Icinga / icinga2

The core of our monitoring platform with a powerful configuration language and REST API.
https://icinga.com/docs/icinga2/latest
GNU General Public License v2.0
1.99k stars 571 forks source link

Prometheus remote writer #9900

Open p4k8 opened 9 months ago

p4k8 commented 9 months ago

Is your feature request related to a problem? Please describe.

Problem is storing perfdata in a Prometheus-compatible storage (Victoriametrics, Cortex, Thanos, Mimir etc) for it to be compatible with other monitoring tools while being cost-efficient.

Describe the solution you'd like

Native Prometheus remote protocol writer, similar to InfluxDB and OpenTSDB perfdata writers.

Describe alternatives you've considered

As of now I employ workaround consisting of writing with InfluxDB writer to the vmagent tool of Victoriametrics suite. It works fine most of the time, but data model is different and it might name and store things in somewhat awkward way due to conversion.

Additional context

https://prometheus.io/docs/concepts/remote_write_spec/ https://github.com/google/snappy/tree/main https://github.com/prometheus/compliance/tree/main/remote_write_sender

martialblog commented 2 months ago

Just some thoughts on this: Instead of a Writer for Prometheus alone it might be even more beneficial to use OpenTelemetry https://opentelemetry.io/

This would allow for sending data to even more metric backends, including Prometheus.

Prometheus will move also move to OTel in the longterm: https://prometheus.io/blog/2024/03/14/commitment-to-opentelemetry/

Edit: Elastic and Influx are on their way to fully support OTel as well. Another argument for an OTel Writer.

Together with tools like the OpenTelemetry Collector or Telegraf this would enable Icinga2 to be extremely flexible for writing Perfdata.