open-telemetry / opentelemetry-python

OpenTelemetry Python API and SDK
https://opentelemetry.io
Apache License 2.0
1.78k stars 623 forks source link

Allow providing a "MetricProcessor", similar to a SpanProcessor #4230

Open nirga opened 2 days ago

nirga commented 2 days ago

Is your feature request related to a problem?

I'm maintaining OpenLLMetry and we have an SDK that wraps the instrumentations and enriches them with different properties. For traces, we use the SpanProcessor to tap into on_start and add attributes to relevant spans. We want to do something similar for metrics (add some attributes), so ideally, we'd love to have a way to do that before a metric is added / aggregated on the OpenTelemetry SDK level.

Describe the solution you'd like

I'd imagine something on the level of the consume_measurement - potentially allowing to specify a callback / attribute enrichments, or overriding the consumer.

Describe alternatives you've considered

I couldn't find any other way to add attributes to a metric after it's recorded.

Additional Context

No response

Would you like to implement a fix?

Yes

lzchen commented 1 day ago

@nirga

Yes there doesn't seem a very powerful way to configure metrics in the sdk as of today (except for [views](https://github.com/open-telemetry/opentelemetry-python/tree/main/docs/examples/metrics/views but the configuration options are limited). A couple of questions:

  1. Is your use case such that you want the metric to be enriched during collection since some information is only available then? Or do you only care what they wind up being exported as?
  2. Do you know if there is precedence in other languages for introducing a mechanism as powerful as a spanprocessor for metrics? Since this is not speced out, it would most likely be a experimental feature.
nirga commented 1 day ago

@lzchen thanks!

  1. Hmm, not sure I understand the difference? I want the enriched attributes to be available when the metrics "leave" the service.
  2. I haven't seen anything like that indeed, the closest might be on the collector side - https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/processor/metricstransformprocessor/README.md
lzchen commented 5 hours ago

@nirga

Hmm, not sure I understand the difference? I want the enriched attributes to be available when the metrics "leave" the service.

I'm wondering if you can achieve this desired behavior by using a custom exporter and enrich the metrics data with the attributes you want prior to export.

nirga commented 5 hours ago

@lzchen the problem is that I need the context which contains the attributes to enrich - and in the exporter I no longer have the context of the specific execution where the metric was recorded.