search

yaorg / node-measured

A Node metrics library for measuring and reporting application-level metrics, inspired by Coda Hale, Yammer Inc's Dropwizard Metrics Libraries
https://yaorg.github.io/node-measured/
MIT License
517 stars 52 forks source link

Histograms - possible to show windowed averages #25

Closed ryanwilliamquinn closed 7 years ago

ryanwilliamquinn commented 7 years ago

I have a use case where I would like to show the average over different timeframes, rather than simply for all time. Does that seem like something worth adding into the histogram output?

mantoni commented 7 years ago

Sounds like a very specific use case. You can also derive the information from the raw data. Maybe as a separate module?

ryanwilliamquinn commented 7 years ago

Thanks for the quick response. I can see it as a relatively common use case. We ran into it trying to graph http response times.

Maybe I am using the module wrong, but after a couple hours our mean response time was just a straight line: image The blue line is the histogram average, the purple line is a 1 minute moving-average response time for a backend service that is part of the blue response time.

Is the raw data available from the histogram? Or do you mean collecting the raw data separately?

mantoni commented 7 years ago

Ah, I see now. I had a similar use case once. What we did was sending the data to Elasticsearch every 15 seconds and then reset the data. I've built a module for that, in case you're interested: https://www.npmjs.com/package/measured-elasticsearch

Regarding the raw data: There is a toJSON function you can use.

felixge commented 7 years ago

@ryanwilliamquinn I assume you're looking at the mean output of the histogram?

I think that's actually a useless value to look at. I mean it's simply the mean over the lifetime of the histogram object which usually isn't very interesting. Is there a good reason why you're not looking at e.g. the median output? It will be closer to the kind of window'ed average you're trying to compute.

FWIW, all my current monitoring stuff for work is using prometheus. They have a different concept of Histograms where a histogram is essentially a group of counters. The cool thing is that these can be easily aggregated, compute percentiles (with limited error), apdex scores and more. They would also easily support the kind of window'ed averages you're interested in. Here is some more info.

ryanwilliamquinn commented 7 years ago

I was looking at the mean, but switched to the median output, as you mentioned. I will check out prometheus, thanks for the suggestion.