fralalonde / dipstick

Configurable metrics toolkit for Rust applications
Apache License 2.0
99 stars 9 forks source link

Accumulate metrics #82

Open Justarone opened 4 years ago

Justarone commented 4 years ago

Hello, I am a newbie to Rust and have one question about your crate: Is there any way to accumulate count metrics?

For example: if I send metrics every 3 seconds and if I count 10 in first 3 seconds and 20 in second 3 seconds, is there any way to return 30 after 6 seconds? I didn't find that neither in handbook nor in examples... Sorry in advance if question is silly...

fralalonde commented 4 years ago

The current AtomicBucket behavior is for the metrics to reset when they are published. Making this configurable wouldn't be too hard. If the reset-on-publish was disabled, you'd probably still want to reset the metrics at some later point, either manually or on schedule, else the numbers would be evergrowing?

I can see this being useful for things such as an animated progress bar with periodic update... In what situation where you thinking of using this?

fralalonde commented 4 years ago

To be clear, what we are discussing would require a code change to dipstick.

Justarone commented 4 years ago

Hm, sounds fair about "reset-on-publish" especially since there is no loss of information (we can accumulate counters and get what we want on Graphite for example). I just wanted to know is there a possibility to do it via crate, to get needed values on graphite without aggregating metrics there.

In my case I want to count, how many records I'we written in my storage since startup of application. Using your crate I get derivative (read "speed of writing per second") of needed value, and that's not a big problem. As I understand, there is also no problem in ever-growing numbers, because if we use, for example, u64 and count 100.000 per second we need 5 miillion years to overflow it:)

Justarone commented 4 years ago

So I would be pleased, if you'll add this feature somewhen (as I said higher: in many cases ever-growing numbers are not a problem and sometimes we do not need to reset values at all). Maybe I'll make a pull request later, but I am not sure that I have enough experience to do that in right way. Thanks for support!

fralalonde commented 4 years ago

I've just started a new job and am currently swamped with stuff to learn so I may not have time for it for a while. I you want to give it a go, I agree with adding the feature and can guide you towards an acceptable implementation in a collaborative PR. Most important is that the current default should be respected (i.e. don't break existing code). You would need to add a no_reset(true) (I dont like that name but cant think of a better one for now) config flag to AtomicBucket. That flag would be read at the moment of flushing to determine wheteher to clear or preserve the aggregated values. Then, in second phase, you'd need to add a reset method to AtomicBucket that would zap the values.

Justarone commented 4 years ago

I am also swamped now with university and work, so I am not sure that I'll start this soon. But that is cool experience, so I'll try in my freetime for sure:)