search

prometheus / client_golang

Prometheus instrumentation library for Go applications
https://pkg.go.dev/github.com/prometheus/client_golang
Apache License 2.0
5.4k stars 1.19k forks source link

Reset functionality for Counters #1624

Open vista- opened 1 month ago

vista- commented 1 month ago

I think it would be a welcome addition if there was a way to reset a Counter to 0 value without having to resort to workarounds (e.g. unregister and delete a Counter metric, and then recreate it from scratch).

A use-case for this is tracking individual user metrics which are session oriented, i.e. when the user session is recreated/reset, the relevant metrics are also reset to 0. Prometheus' built-in handling for breaks in monotonicity specifically for Counter metrics is very useful when performing analysis and monitoring, and it would be useful to leverage this for Counter metrics which might have multiple sessions during the lifetime of an application, without having to resort to hacks like the ones described above.

The specific use-case in my case is the following: I am writing a RADIUS server, which also handles processing traffic accounting. These traffic statistics (packet and byte counters) increase monotonically per user session. User sessions are handled "per login", which correspond to the user equipment going online and offline from the network.

I have looked at some alternatives to properly represent these traffic statistics as Prometheus metrics:

Gauges

User traffic metrics can be correctly represented, however, breaks in monotonicity are not handled gracefully by Prometheus rate/increase/linear interpolation functions.

Use a different label value per session

This would be the 'cleanest' implementation, requiring no changes to the Prometheus client library, however, this places additional burdens on the developer, such as: indicating which label value is the 'active' session for the user, and pruning the metrics with the old label sets for the user. This would also not preserve metric history across user sessions and contribute to an even higher cardinality per each metrics.

ConstMetrics

As this is effectively mirroring a metric from an external system, this would be the recommended implementation. However, compare implementing a Custom collector with using promauto with a CounterVec, which would be the path of least resistance and the most documented solution for tracking multiple counter metrics with different labels. ConstMetrics are also under-documented (in my opinion).

Implement a custom Counter/expand the Counter metric type

Since the underlying counter implementation of the Counter interface is not exported, it is not possible to easily expand the existing implementation with reset functionality. This would require a fork or my own implementation of a Prometheus client, increasing maintenance burden.

I don't see many downsides of adding reset functionality: resetting Counter values is gracefully handled by Prometheus, breaks in monotonicity are also accounted for and properly handled in functions that calculate over a time series, and this already happens when a Counter metric is reinitialised without a starting value after an application reboot.

I'm happy to contribute the functionality in a PR if this gets accepted as a feature following discussions.

Thanks for considering this new feature, looking forward to hear your thoughts about this!

dosubot[bot] commented 1 month ago

Hello @vista-! I'm here to help you with bugs, questions, and becoming a contributor. Let's work on this issue together.

There are currently no existing discussions or issues related to adding reset functionality to Counters in the client_golang repository [1]. However, the idea of adding reset functionality to Counters is interesting and could be beneficial for session-based metrics, as you described.

Here are some points to consider:

  1. Prometheus Design Philosophy: Prometheus Counters are designed to be monotonically increasing. Introducing a reset functionality might go against this design principle and could lead to unexpected behavior in some use cases.

  2. Workarounds: As you mentioned, unregistering and re-registering metrics is a common workaround, but it has performance implications, especially in high-cardinality scenarios [2].

  3. Custom Implementation: Implementing a custom Counter or expanding the existing Counter metric type would require significant changes and maintenance, as the underlying counter implementation is not exported.

  4. Alternative Approaches: Using different label values per session or Gauges are alternatives, but they come with their own set of challenges, such as handling breaks in monotonicity and increased cardinality.

Given these considerations, it might be worth discussing this feature request with the Prometheus community to gather more feedback and explore potential solutions. If the community agrees on the need for this feature, contributing a PR with the reset functionality could be a valuable addition to the client_golang library.

To continue talking to Dosu, mention @dosu.