Allow multiple metrics exporter (Prometheus, OpenTelemetry)

[thepabloaguilar]

Problem

Currently Flipt uses the OTel library to build up the application metrics but just provide the Prometheus exporter! Depending on the company guidelines it can be a "problem" if they use a APM/Metrics platform (New Relic, Datadog) to store and compute metrics and they want to be vendor-free by using only OTel stack

Ideal Solution

The ideal solution would be to allow people chose what kind of exporter they want and keeping the current behavior for backward compatibility! Something like:

metrics:
  prometheus:
    enabled: true # It should be the default value for backward compatibility
  otlp:
    enabled: true
    endpoint: "https://{your-collector-url}"

Search

• [X] I searched for other open and closed issues before opening this

Additional Context

Related #2978

[markphelps]

Great idea @thepabloaguilar !!

[markphelps]

One thing to note is that we arent currently using the OTEL prometheus exporter, we are using the prometheus lib itself to capture metrics and 'export' them using the promhttp lib mounted at /metrics

I think in order to cleanly accomplish the goal of this issue, we would need to change out the metrics collection APIs to use the OTEL lib instead of the prometheus lib itself, something like they do here in the prometheus OTEL exporter example:

https://github.com/open-telemetry/opentelemetry-go/blob/b34cfc47c4e0240b728f28e62f829dc25408746b/example/prometheus/main.go#L30-L55

[GeorgeMac]

One thing to note is that we arent currently using the OTEL prometheus exporter, we are using the prometheus lib itself to capture metrics and 'export' them using the promhttp lib mounted at /metrics

I think in order to cleanly accomplish the goal of this issue, we would need to change out the metrics collection APIs to use the OTEL lib instead of the prometheus lib itself, something like they do here in the prometheus OTEL exporter example:

https://github.com/open-telemetry/opentelemetry-go/blob/b34cfc47c4e0240b728f28e62f829dc25408746b/example/prometheus/main.go#L30-L55

We moved to the otel metrics API and prom provider for it a while back now. This is the thing than broke every week because the API was unstable for so long. But it’s all good now, we’re good to go with other otel provider impls for metrics.

https://github.com/flipt-io/flipt/blob/2490bb5ef7d767000b50f5faf26c1f99c23ed0d1/internal/metrics/metrics.go#L23

[markphelps]

Oh nice! Should be a pretty small lift then to get support for the OTLP exporter then. I can create a PR for this tonight!

Thanks for the reminder @GeorgeMac