search

deadtrickster / prometheus.erl

Prometheus.io client in Erlang
MIT License
341 stars 119 forks source link

Full summary implementation #18

Open deadtrickster opened 8 years ago

deadtrickster commented 8 years ago

Now only basic variant of summary metric is implemented - with sum and count of observations.

What's left:

Prometheus summaries usually use highly biased quantile and since we deal with streaming data so they always implement the same algorithm from paper [0]. Example implementations: [1-3].

Sliding window seems to be implemented using N numbers of age-overlapped summaries maintained in parallel. Of course only 'top' one is rendered during scraping.

Due to the huge overhead maybe it's good idea to keep current simple implementation as an option?

References:

  1. Effective Computation of Biased Quantiles over Data Streams
  2. Ruby Quantiles implementation
  3. Golang Quantiles implementation
  4. Common Lisp implementation
  5. Golang sliding window implementation
hairyhum commented 3 years ago

So I found this: https://github.com/odo/quantile_estimator It's rather old but seems to build, at least on OTP-21. I'll take a look if it's possible to integrate to the summary metric.