ITS
TIME
TO
TAKE
TIME
TO
TAKE
TIME
INTO
ACCOUNT
ITS
TIME
TO
TAKE
TIME
TO
TAKE
TIME
INTO
ACCOUNT
So this talk is about Time Series Databases AKA Temporal Databases you might see them referred to as TSDB or TSDS
Now if your job is very boring (or perhaps on the contrary extremly exciting!) you may have met those as "Operational Historians or Enterprise Historians "
This very short talk is a public service of sorts from the BuzzProphets.
Those of you who know me know I have my ear firmly stuck to the ground.
OH probably the vast majority doesn't know me so...
Hello my name is Ori Pekelman. I am OriPekelman everywhere (twitter/github/linked-in).
My blog is on http://blog.constellationmatrix.com
My company, Constellation Matrix, does software architecture and I made myself a nice logo.
Back to our business of holding one's ear to the ground in a very uncomfortbale position. And to TSDS's
So a Time Series Database is a speciality thing, a domain specific piece of software. These has been around for a while.
You can see them in control and industrial systems to record every last damn thing that is happening.
In financial systems, in energy. Companies like GE and Honywell produce this kind of software.
Something that has a boring yet intimidating web page like this
There are even guys in this domain that get to have a 2 letters .com domain.. http://kx.com
So what kind of data is Time Series Data?
OPTION A: Web Logs
OPTION B: Stock Prices
OPTION C: Everything
I will posit that all data is Time Series Data.
This is a very short talk so I can't really delve into this. But consider that the only operation we know how to do, reliably, in a distributed system is insert.
Because we only know how to insert
We can basically always suceed in writing somewhere
User A has Expressed Intent B on Data Piece C
your operations or binary log, your write side cache, your append only file.. whatever.
But this information is basically useless if we don't put this in a timeframe. User
A has Expressed Intent B on Data Piece C at moment D
If I can't convince you all data is time series data
At least you'd agree that all "big data" is time series data. Right?
SOO....
TIME IS A BIG THING
And time is I believe going to have a very good 2014
Did you see how all the colors disappeared?
And Google doesn't show interest...
Yet I think that everybody is kind of going to wake up in the coming months slapping their fronts and saying, we are all implementing at a stupidly low level the same patterns over the same types of data. Can't we get something off the shelf for this?
I got interested in this while thinking about the following: On an e-commerce site, how do I identify, in real-time, customers that are hesitating. Considering they have probably multiple tabs open.
You can see this emerge already with the very intersting way ElasticSearch is developing (look at Kibana3 if you haven't)
Very recently druid.io went out
Look at what is a first class object with them
This is the canonical example on their homepage
More of the purists, look at https://tempo-db.com
You can see this also in very interesting projects like http://skydb.io/ and this will probably be even a stronger trend, Time Series Databases for behavioural data.
Because the whole big data thing... well its around that right?
Keen.io is another service geared specifically to analytics.
You might also want to have a look at the two years old MongoDB based solution from Square Cube http://square.github.io/cube/
There are of course OpenTSDB and Kairo (respectively for Hbase and Cassandra) we even see emerging the "as a service crowd" (have a look at TempoDB, Keen) on this segment.
You know its cool when even the Zindoz guyz have one http://openhistorian.codeplex.com/
What makes a Time Series Databases a cool thing?
- Something that distnguishes the ephemeral from the stable (Intent and Ontos)
- Something that recognises human time is both linear and circular in nature and that the most interesting artefacts are those that repeat..
- Something that knows that you agregate over a second for the last hour over a minute for the last day, over an hour for a month...
- And if you agree that all data is time series data then the only thing that can repersent the "universe" or reality is a time series database.
BECAUSE NATURE DOES NOT HAVE MUTABLE STATE
Ultima Necat.. Well I ran out of time...
I am OriPekelman everywhere (twitter/github/linked-in).