OpenTelemetry: implement a mapping from ZIO Metrics Summary to OTEL Histogram

zio / zio-telemetry

ZIO-powered OpenTelemetry library

https://zio.dev/zio-telemetry

Apache License 2.0

112 stars 55 forks source link

OpenTelemetry: implement a mapping from ZIO Metrics Summary to OTEL Histogram #821

Open grouzen opened 4 months ago

grouzen commented 4 months ago

This functionality wasn't implemented in https://github.com/zio/zio-telemetry/pull/801 as it is tricky to do.

As a reference you can look at the https://github.com/micrometer-metrics/micrometer library. It does exactly this for OTEL integration.

IvanFinochenko commented 4 months ago

Hi @grouzen What do you think can we use OpenTelemetryMeterRegistry and DistributionSummary.Builder for this issue?

IvanFinochenko commented 4 months ago

I have researched some libraries and find this OpenTelemetryMeterRegistry It has DistributionSummary, but methods for building OpenTelemetryMeterRegistry are not flexible. I will try to create a PR with method builder taking OpenTelemetry Meter

grouzen commented 4 months ago

Hi @IvanFinochenko! Sorry, don't have enough time at the moment to check this. Regarding OpenTelemetryMeterRegistry - I don't think we need it since we maintain our own registry for our instruments. We just need to "copy" the way they map DistributedSummary to OTEL Histogram. The problem is the parts of the mapping algorithm are scattered all over the micrometer's codebase so it is tricky to understand it as a whole.

grouzen commented 4 months ago

@IvanFinochenko We need to implement https://github.com/zio/zio-telemetry/blob/series/2.x/opentelemetry/src/main/scala/zio/telemetry/opentelemetry/metrics/internal/OtelMetricListener.scala#L40. For this, we need to use registry.getHistogram and transform the payload of the ZIO Metrics Summary into OTEL Metrics Histogram. Hope it makes sense.

grouzen commented 4 months ago

@IvanFinochenko, we could do a pairing session in Discord, for example, if you want.

IvanFinochenko commented 4 months ago

@grouzen it is a good idea. I need a little more time to dive deeper under hood of micrometer and I'll write you.

andrzejressel commented 1 month ago

So I also took a look at this and I found following things:

Summary is marked as deprecated/do not use. It is supported directly with Protobufs, but not in Java API
Micrometer does not use OTEL Java library - so they can just push summaries without issues.
Summaries are not really transferrable to histograms - the issue with summaries is that their timeline is weird - they are not just values they are precomputed values.

I personally see 2 solutions

Push to Pull

Currently in zio-telemetry metrics are added directly to Java API counterparts. Instead of doing that we could utilize MetricProducer to instead pull metrics from inside ZIO-Metrics.

Deprecate summary in ZIO

From my understanding OTEL is like a consorcium of companies that does telemetry stuff. If they decided that summary metric is deprecated, should we still have it?

grouzen commented 1 month ago

@andrzejressel Hey! I want to clarify this one. From your reply, I understood you are talking about adding support for OTEL Summary, which is legacy. This particular issue is about mapping ZIO metrics Summary to OTEL Histogram because OTEL Summary is deprecated, as stated above. That's precisely what the micrometer library does internally. It also avoids dealing with OTEL Summary and uses quite complex logic to map summaries to histograms for the OTEL case.