Open bwplotka opened 4 months ago
Opinionated disclaimer: Using OTLP push (or generally push) has its own disadvantages. Converting Prometheus metrics to OTLP format has another set of disadvantages. Our recommendation is to prefer pull mechanisms for metric pipelines for reliability, centralised discovery, efficiency and simple client failover scenarios. Standard
promhttp
/metrics pull mechanism remains the most efficient and reliable way to directly get metrics from your application to Prometheus or Prometheus compatible backend.
I can't emphasize this point more
but wish to use OTLP push instead of (or additionally to)
promhttp
/metrics pull method, because perhaps it's a batch job or you want to use push based collection mechanisms with e.g. OTel collector
In theory, this approach would eliminate the use of https://github.com/weaveworks/prom-aggregation-gateway?tab=readme-ov-file (or maintained version https://github.com/zapier/prom-aggregation-gateway)
@bwplotka were you thinking of an example, or just documentation? If documentation, where do you think this belongs?
Thanks to amazing community behind OpenTelemetry Go SDK (special kudos to @dashpole), users of client_golang can optionally import a new Go module, a special bridge [1][2] and push data from
prometheus.Registry
using OTLP push.This is useful if you have your application metrics instrumented with Prometheus client_golang, but wish to use OTLP push instead of (or additionally to)
promhttp
/metrics pull method, because perhaps it's a batch job or you want to use push based collection mechanisms with e.g. OTel collector. Notably, initial benchmarks shows that, currently, this way is more efficient than e.g. pulling metrics using Otel collector Prometheus receiver and converting to OTLP data model. This situation, however, can change as the source of the efficiency issues can be, in theory, optimized away.Opinionated disclaimer: Using OTLP push (or generally push) has its own disadvantages. Converting Prometheus metrics to OTLP format has another set of disadvantages. Our recommendation is to prefer pull mechanisms for metric pipelines for reliability, centralised discovery, efficiency and simple client failover scenarios. Standard
promhttp
/metrics pull mechanism remains the most efficient and reliable way to directly get metrics from your application to Prometheus or Prometheus compatible backend.Nevertheless for those who wants to leverage pure OTLP, this issue is to track adding relevant explanation to our docs. This is also on part with Java Prometheus client offering OTLP