zapier / prom-aggregation-gateway

An aggregating push gateway for Prometheus
Mozilla Public License 2.0
116 stars 26 forks source link

Batch gauge data to create more realistic gauge projections for a given polling interval #6

Open mplachter opened 1 year ago

mplachter commented 1 year ago

As a user, I would like to be able to Batch incoming Gauge data and then on a interval compute the sum and set it as the correct aggregate for that given resolution period.

sontek commented 1 year ago

@mplachter Did you make this change? I know you were working on some gauge work

mplachter commented 1 year ago

I did not get around to this one yet. Right now all gauge data is still just aggregated together

sontek commented 1 year ago

This fork did the work, maybe we can pull it in?

mplachter commented 1 year ago

It looks like he took an average gauge value approach 🤔

I think that's half of the equation looks like they always keep values and keep averages going... I think over a long period of time this will not be the fairest representation of the gauge data for example.

Last AVG was 10 new data coming in is 100 + 100 + 100 it will say the gauge average 88

(10 + 100) / 2 = 55 
(55 + 100) / 2 = 77
(77 + 100) / 2 = 88

But the real average during that gauge interval should have been 100.

We still do need the code to average them out, I think we definitely should look at the performance of the fingerprinting implemented in the above implementation.