Micrometer Observation Support

[marcingrzejszczak]

I'm a co-maintainer of Spring Cloud Sleuth and Micrometer projects (together with @shakuzen and @jonatan-ivanov).

Micrometer Observation is part of the Micrometer 1.10 release and Micrometer Tracing is a new project. The idea of Micrometer Observation is that you instrument code once but you get multiple benefits out of it - e.g. you can get tracing, metrics, logging or whatever you see fit).

Since this project already has tracing capabilities I was curious if there's interest in adding Micrometer Observation support so that except for metrics, spans could be created and tracing context propagation could happen too. Via Micrometer Tracing one can use OpenTelemetry or OpenZipkin Brave Tracer, but with the handler mechanism the possibilities are endless :)

If there's such interest we could provide a PR to add support for that.

[marcingrzejszczak]

From what I see Helidon has an inbuilt tracing functionality. It resembles a lot what I did with Apache Camel instrumentation. To add Micrometer Observation support we would need to do something very similar

• add high and low cardinality tagging option in the current api (backward compatible change) , example for Camel. This is required because otherwise the metrics created from Observations would have no tags (cause they get the low cardinality tags only)
• add dedicated contexts for sender and receiver, example for Camel
• in Micrometer Observation setting of propagation technique is done not via an inject method but in the context itself. Camel had the same approach as Helidon has where those are separated. We would need to do it slightly differently, example for Camel

[tomas-langer]

Hello, thanks a lot for this issue and the offer!

I will investigate the links you have provided and decide with the team on further action.

A "short" summary of what we have (and plan to do):

What we have for integration with metrics, tracing, logging

1. we have integration with Micrometer for metrics (with our own agnostic APIs being prepared for 4.x)
2. we have integration with OpenTracing and OpenTelemetry for tracing (with our own agnostic APIs)
3. we support System.Logger (4.x), and have integrations with Java Util Logging (3.x is based on it), log4j, and slf4j, we also have an MDC support for all three (custom built for JUL)
4. we currently do not do an end-to-end correlation of all of these (can be achieved through MDC)
5. we have (and are working on further) cloud specific integrations, which I think are not relevant for this conversation

Related to "how" we can integrate things:

1. Helidon has two "flavors" - you can do an "imperative" programmatic mode, and a "declarative" mode (e.g. "I create everything by hand and send it around myself" vs. "I Inject stuff from a service registry, and I can intercept annotated methods") - our integrations should work in both worlds (this makes the work a bit harder, as in addition to interceptors/annotation, we have programmatic APIs that can be used from anywhere).
2. We are working on our own service registry implementation (in additon to CDI we already support for MicroProfile), which will also support injection and interception - our intention for the future is to have a single set of APIs usable in all three worlds

[jonatan-ivanov]

Hi,

I think The Observation API can be interesting for you from the perspective of #1, #2, maybe #3 (depending on your needs), and #4. Since it provides a set of tools to instrument your code and then get the data you want out of it (metrics, tracing, logs, etc.), you can consolidate the Micrometer Metrics and the OpenTelemetry support (and other things too) under one API. This means that if you use the Observation API, you will get metrics, tracing (through OTel or Brave) and other signals without instrumenting the code multiple times with multiple APIs. (I'm not sure what you want to do with OpenTracing since it never went GA and it is already deprecated/abandoned.)