fluent / fluent-plugin-prometheus

A fluent plugin that collects metrics and exposes for Prometheus.
Apache License 2.0
257 stars 80 forks source link

Expose only topk metrics to Prometheus #205

Open phihos opened 2 years ago

phihos commented 2 years ago

When having a lot of label combinations for a metric but only the metrics with large values are of interest (e.g. a url label) it might make sense to restrict the metrics presented by the HTTP server only to "topk" values with k being configurable on a per-metric basis. I think this only makes sense for gauges and counters.

I already looked up how it can be done and I am willing to implement it including tests. For that to work we need to introduce and own data store based on the default data store. In it we would need to add a metric setting "topk" which makes all_metrics return just the top k metrics instead of all. If topk is not set all_metrics should return all metrics. Finally we need to set that data store as default in the plugin code and add a topk option when initializing a metric.

Is that something you would consider accepting into your code base?