search

c-base / cbeam-telemetry-server

c-beam telemetry provider for NASA OpenMCT
https://bergie.iki.fi/blog/nasa-openmct-iot-dashboard/
61 stars 14 forks source link

Downsample historical data #54

Open bergie opened 7 years ago

bergie commented 7 years ago

Currently we always store datapoints as they come in, and query at full resolution. With certain kinds of sensors this can become heavy.

We should implement downsampling for historical data, and set a retention policy on the full resolution tables

https://docs.influxdata.com/influxdb/v1.3/guides/downsampling_and_retention/

baccenfutter commented 7 years ago

Why don't you simply pump the data into RRDs? I mean, thinning out metrics is a problem that has been solved generically pretty well. Why reinvent the wheel?

EDIT: Ah, my bad. The sole idea is to use InfluxDB, I take it.

jdbodyfelt commented 4 years ago

You can possibly handle the downsampling directly in the InfluxQL query. For example, downsampling to reading only every 30 seconds would look something like:

SELECT FIRST(field) FROM database..measure
WHERE time >= start AND time <= end
GROUP BY time(30s)