open-telemetry / opentelemetry-java-instrumentation

OpenTelemetry auto-instrumentation and instrumentation libraries for Java
https://opentelemetry.io
Apache License 2.0
1.79k stars 782 forks source link

"info" str of metric name will be removed #11658

Open halleystar opened 6 days ago

halleystar commented 6 days ago

Describe the bug

if i have a metric name which name is "dubbo_application_info_total". the result will be changed to "dubbo_application__total" without "info".

Steps to reproduce

when use myself extension for collecting metric of dubbo3. I found the not expected behavior.

Expected behavior

info str will be reserved?

Actual behavior

info str was removed?

Javaagent or library instrumentation version

Javaagent 2.4.0

Environment

JDK: OS:

Additional context

No response

laurit commented 5 days ago

This isn't a built in metric of otel agent. How do you get it? Is it so that dubbo reports it with micrometer, you have enabled otel micrometer instrumentation, you export the metric to prometheus? As far as I understand dubbo_application_info_total is fine as far as otel is concerned but when exporting metrics to prometheus the conversion happens in https://github.com/open-telemetry/opentelemetry-java/blob/cbac2020442434a8d00dcb24d60def318b1973bd/exporters/prometheus/src/main/java/io/opentelemetry/exporter/prometheus/Otel2PrometheusConverter.java#L536 that calls sanitizeMetricName https://github.com/prometheus/client_java/blob/5867b799a735288147bfd9ee680ca588a34490b9/prometheus-metrics-model/src/main/java/io/prometheus/metrics/model/snapshots/PrometheusNaming.java#L132 that removes _total and _info suffixes.

apanagiotidis-soundhound commented 3 days ago

This also caught me by surprise, but it is actually due to a change in Micrometer. They recently made some metric name suffixes invalid, see https://github.com/micrometer-metrics/micrometer/wiki/1.13-Migration-Guide#invalid-meter-suffixes

I believe this change was made to comply with OpenCensus, see https://github.com/OpenObservability/OpenMetrics/blob/main/specification/OpenMetrics.md#suffixes

In my short investigation I was not able to find a way to turn that off, so we ended up renaming our metrics.