tikv / rust-prometheus

Prometheus instrumentation library for Rust applications
Apache License 2.0
1.05k stars 182 forks source link

Add MaximumOverIntervalGauge #469

Open JanBerktold opened 1 year ago

JanBerktold commented 1 year ago

Depends on #405.

This is another "fancy" gauge type from other internal codebase that I believe to be beneficial to others.

We have a few use cases where we want to monitor a value that moves around a lot. One example is measuring the number of concurrent requests for a gRPC server. With a normal gauge, we're only able to inspect the value as of the time of the scraping (typically once every 10 seconds). This actually got us in production -- We had queue build ups due to downstream TCP re-connects that were manifesting themselves as increased latency for our customers but our metrics looked fine.

Enter the MaximumOverIntervalGauge: Instead of reporting the current value, we report the maximum of the value over the last (user configured) interval. This allows us to observe shifts that resolve themselves in less than the configured scrape interval.

lucab commented 1 year ago

Interesting, I'll need to read a bit more about the underlying logic. Does this have an equivalent in Go world (or any other client library)?