aio-libs / aiokafka

asyncio client for kafka
http://aiokafka.readthedocs.io/
Apache License 2.0
1.13k stars 228 forks source link

Add compability with OpenTelemetry #862

Open Corfucinas opened 1 year ago

Corfucinas commented 1 year ago

Describe the solution you'd like Currently, there's no clear way to export metrics or traces running directly from the program. Opentelemetry allows auto instrumentation with this python package. If we use sync kafka-python it's possible to export traces by just adding KafkaInstrumentor().instrument(), in Jaeger this is how they look.

image

However, this will not work with the current implementation of aiokafka

I could start the conversation over there as well, however, I think they won't be able to include aiokafka since it -unfortunately- does not exposed metrics

marcosschroh commented 1 year ago

Hi!! Any news on this?

ods commented 1 year ago

What features do you miss on aiokafka side? KafkaInstrumentor from the link above just monkey-patches two methods of producer and consumer, I see no problem to do the same for aiokafka's producer and consumer.

AMontagu commented 1 year ago

Hello

Had the same issue at my company today. Decide to adapt the interceptor for aio. Please note that I have just started it this afternoon and it may not work in all cases: https://github.com/socotecio/opentelemetry-instrumentation-aiokafka

Will transform this to a real package when time but you can download it and use it since now.