rcrowley / go-metrics

Go port of Coda Hale's Metrics library
Other
3.43k stars 493 forks source link

Unregistering depleted exponential decay/EWMA counters #283

Open davidstrauss opened 3 years ago

davidstrauss commented 3 years ago

I'm working on a system that may have a very large number of ExpDecaySample/EWMA counters, but most will quickly go to zero. That is, I plan to increment a large breadth of them, but most will decay to zero fairly quickly. I'd like to avoid having them pile up, especially since I can lazily reinitialize them.

One thing I'm looking at is using Each on a registry to Unregister decaying counters that have hit zero. I worry this is unnecessarily aggressive, though, as there's little harm in keeping around counters that have hit zero other than the goal of "garbage collection" for them. Also, I'm not sure if it's safe to unregister a metric that is currently passed to me via Each. Unlike for Register and GetOrRegisterCounter, I don't see any notes in the docs about thread safety (or other expectations).

Alternatively, I'm curious about options for discarding counters using an LRU/LFU when under memory pressure (or hitting other defined limits). I'm not particularly worried about empty counters taking space until the space becomes a problem.