heubeck / samler

SaMLer - Smart Meter data collector at the edge
GNU General Public License v3.0
5 stars 3 forks source link

Aggregate by time #4

Open heubeck opened 1 year ago

heubeck commented 1 year ago

Especially with MySQL and similar backends the provided data might be too much. A configurable value to report aggregated values (like average over 5 minutes) less frequent would be great. Avg must not be senseful for every measurement, 'last' is more appropriate for e.g. a counter.

seehase commented 1 year ago

Aggregation period should be configurable e.g. in minutes

If there is an aggregation please provide additional information for "counters" like ident 1.8.0 time, powermeterid, ident, value, unit, avg, max, min e.g. avg is calculated (counter_new - counter_old) / interval

                                       (38075)
time               ;   id     ident;   ;val  ;   unit  ;avg     ;max     ;min     ;unit
11.07.2018 15:30:00;   IDxyz  1.8.0    ;38085;   kWh;   2,000;   3,134;   1,234;   kW   
11.07.2018 15:35:00;   IDxyz  1.8.0    ;38093;   kWh;   1,600;   1,808;   0,934;   kW
11.07.2018 15:40:00;   IDxyz  1.8.0    ;38107;   kWh;   2,880;   3,432;   2,456;   kW
11.07.2018 15:45:00;   IDxyz  1.8.0    ;38125;   kWh;   3,600;   5,234;   1,345;   kW
seehausen commented 1 year ago

I think as long as we use influxDB, aggregation is not needed A better approach could be influx downsampling to store reduce historical data in a second bucket and use union(tables: [power0, power0_downsampled]) to show all data in a single graph

https://www.influxdata.com/blog/downsampling-influxdb-v2-0/

sample query:

power0 = from(bucket: "power0")
  |> range(start: v.timeRangeStart, stop: v.timeRangeStop)
  |> filter(fn: (r) => r["_measurement"] == "power")
  |> filter(fn: (r) => r["ident"] == "15.7.0")

power0_downsampled = from(bucket: "power0_downsampled")
  |> range(start: v.timeRangeStart, stop: v.timeRangeStop)
  |> filter(fn: (r) => r["_measurement"] == "power")
  |> filter(fn: (r) => r["ident"] == "15.7.0")

union(tables: [power0, power0_downsampled])
  |> aggregateWindow(every: v.windowPeriod, fn: mean, createEmpty: false)
  |> yield(name: "last")