search

WebThingsIO / gateway

WebThings Gateway
http://webthings.io/gateway
Mozilla Public License 2.0
2.61k stars 333 forks source link

Add configurable smoothing/averaging/sampling to logs to limit amount of data written #2082

Open hobinjk opened 5 years ago

hobinjk commented 5 years ago

Something like "sample rate: 1min" would be good. I'm not sure what would be the most user-friendly options here, i.e. if people would want average over that minute, all data to be dropped for 1 minute, or another variation.

tim-hellhake commented 5 years ago

Sound a bit like solving problems which are already solved. Timeseries databases like influxDB allow you to continuously downsample older data to save space while preserving high resolution for the near past.

hobinjk commented 5 years ago

Yes, this is absolutely intended to be reinventing the wheel. One of the priorities of our current logging system is to not add dependencies to cut down base image size (e.g. influxDB is 100MB) and keep our build process simple

flatsiedatsie commented 5 years ago

It sounds to me like the Gateway needs a general settings page.

hobinjk commented 5 years ago

Yes, definitely! But in this case I would expect the setting to be per-log since the retention window is already an equivalent setting done for each log. I also think that while someone might not care about capturing the temperature more than once an hour, that same person might want to have second-resolution information about power fluctuations.