search

dashbitco / broadway_kafka

A Broadway connector for Kafka
222 stars 51 forks source link

Implementing offset lag telemetry #145

Open David-Klemenc opened 2 weeks ago

David-Klemenc commented 2 weeks ago

Would it be possible to get lag offset in telemetry something like kolt ?

v0idpwn commented 2 weeks ago

I think you should be able to use kolt with broadway_kafka as it uses brod

David-Klemenc commented 2 weeks ago

I can get producer names from Broadway:

Broadway.producer_names(:kafka_test)
[:"kafka_test.Broadway.Producer_0"]

And from the logs I know that the client names look like

:"Elixir.kafka_test.Broadway.Producer_0.Client"

But I have not found a way to get the client name from a Broadway api ? (I need it to call :brod)

:brod.fetch_committed_offsets(:"Elixir.kafka_test.Broadway.Producer_0.Client", "test_group_id")
josevalim commented 1 week ago

I don’t believe there is one. A PR is welcome!