Allow reactive messenging connectors to create consumer spans

[wabrit]

Description

When tracing inter-microservice interactions it is essential to be able to propagate tracing metadata across synchonrous and asynchronous calls. In the latter case, while Quarkus reactive messaging connectors create a span representing the connector processing of the message, they do not have the capability to create a span, descended from the incoming message span, that the consumer can enlist in.

This means that e.g. a message consumer which then makes a REST client request does not automatically result in the span for that request being a descendant of the span associated with the inbound message, effectively breaking (from a tracing perspective) the causal releationship between the two interactions.

While it is possible for application code to manually create the child span in the incoming message handler, it would reduce the amount of boilerplate code if this could be an optional feature of the connector itself.

Implementation ideas

The following is based on some experiments and conversations with @kenfinnigan and @cescoffier :

• Span creation (and starting) can be done within the connector, with any inbound message span as the parent.
• Span.makeCurrent() can then be called to arrange for this to be the current span for the associated Vertx context (via the QuarkusContextStorage implementation).
• The Span can be closed in the ack/nack callbacks on the message; however if the ack chain is cut (because of pre-processing), it may not capture the end of processing. (Also provision needs to be made within the connector for the scenario in some messaging technologies where consumer ack is automated.)

This approach unfortunately does not work as expected, because there is no guarantee that the consumer is bound to the same Vertx context; if it is not then the current span associated with that context will not have been set from the connector code. It is possible that connector and consumer are running on different Event Loop Threads as well.

In order for this approach to work, it might require that the opentelemetry Context is set immediately before the processing method is executed (a concern may be the overhead incurred by that, even when you want tracing, so it may need to be an additional optional feature on top of basic tracing).

[geoand]

cc @kenfinnigan @cescoffier

[quarkus-bot[bot]]

/cc @Ladicek, @cescoffier, @michalszynkiewicz, @phillip-kruger

[radcortez]

Closing in favor of https://github.com/quarkusio/quarkus/issues/22906.