me-box / zestdb

ZestDB
MIT License
18 stars 3 forks source link

Restrict data types allowed in a time-series post #10

Closed jptmoore closed 6 years ago

jptmoore commented 6 years ago

Currently, any JSON can be associated with a timestamp.

I think it makes more sense to restrict to integer and floats as this will allow for more efficient data handling.

Not sure if it makes sense to have a timestamp per data point or allow multiple data points to be associated with a timestamp eg.

[ (1511254543330, 1), (1511254543330, 2), (1511254543330, 3) ]

vs

1511254543330 => [1, 2, 3]

cgreenhalgh commented 6 years ago

There are use cases for both timeseries of primitive values which can be further optimised (e.g. from sensors) but also for timeseries of complex values (e.g. tweets, messages) which could not be optimised in the same way. See also me-box/databox#194 .

So either there are two slightly different timeseries APIs or more than one store type or some one-time per-datasource initialisation of the store. I guess two slightly different APIs might be better/safer.

Toshbrown commented 6 years ago

KV supports three types of data TEXT, JSON and BINARY

The time series endpoint only supports JSON.

My suggestion is to add a NUMERICAL data type to the time series endpoint. This would then allow for the use of numerical only endpoints for example /ts/[dsid]/mean and /ts/[dsid]/min/[from]/[to].

The exact aggregation operations supported would need to be specified.

Some examples of the types of aggregations that are used with time series data can be found here https://docs.influxdata.com/influxdb/v0.8/api/aggregate_functions/ (this is old but it's a nice list )