search

seglo / kafka-lag-exporter

Monitor Kafka Consumer Group Latency with Kafka Lag Exporter
Apache License 2.0
640 stars 188 forks source link

Exporter no longer compatible with Kafka broker 2.x (due to Kraft) #461

Open tgaudillat02 opened 1 year ago

tgaudillat02 commented 1 year ago

Describe the bug Since the Kafka connector was update to 3.x, the exporter is no longer compatible with broker 2.x due to Kraft connection. Here the error during connexion : 

2023-04-28 13:11:32,933 DEBUG o.a.k.clients.admin.KafkaAdminClient akka://kafka-lag-exporter/user/consumer-group-collector-perf3 - [AdminClient clientId=adminclient-2] Call(callName=describeGroups(api=FIND_COORDINATOR), deadlineMs=1682687502932, tries=1, nextAllowedTryMs=1682687493933) failed with non-retriable exception after 1 attempt(s) java.lang.Exception: NoBatchedFindCoordinatorsException: Cannot create a v3 FindCoordinator request because we require features supported only in 4 or later.
    at org.apache.kafka.clients.admin.KafkaAdminClient$Call.fail(KafkaAdminClient.java:819)
    at org.apache.kafka.clients.admin.KafkaAdminClient$AdminClientRunnable.handleResponses(KafkaAdminClient.java:1251)
    at org.apache.kafka.clients.admin.KafkaAdminClient$AdminClientRunnable.processRequests(KafkaAdminClient.java:1416)
    at org.apache.kafka.clients.admin.KafkaAdminClient$AdminClientRunnable.run(KafkaAdminClient.java:1339)
    at java.base/java.lang.Thread.run(Unknown Source)

To Reproduce Deploy a 2.5.0 Kafka broker and try the 0.8.2 exporter version.

Environment

Additional context The update was done in the 0.7.2 version of the exporter.

perdue commented 1 year ago 
2023-04-28 13:11:32,933 DEBUG o.a.k.clients.admin.KafkaAdminClient akka://kafka-lag-exporter/user/consumer-group-collector-perf3 - [AdminClient clientId=adminclient-2] Call(callName=describeGroups(api=FIND_COORDINATOR), deadlineMs=1682687502932, tries=1, nextAllowedTryMs=1682687493933) failed with non-retriable exception after 1 attempt(s) java.lang.Exception: NoBatchedFindCoordinatorsException: Cannot create a v3 FindCoordinator request because we require features supported only in 4 or later.
  at org.apache.kafka.clients.admin.KafkaAdminClient$Call.fail(KafkaAdminClient.java:819)
  at org.apache.kafka.clients.admin.KafkaAdminClient$AdminClientRunnable.handleResponses(KafkaAdminClient.java:1251)
  at org.apache.kafka.clients.admin.KafkaAdminClient$AdminClientRunnable.processRequests(KafkaAdminClient.java:1416)
  at org.apache.kafka.clients.admin.KafkaAdminClient$AdminClientRunnable.run(KafkaAdminClient.java:1339)
  at java.base/java.lang.Thread.run(Unknown Source)

I can confirm this error. I am seeing the same in the logs with any kafka-lag-exporter version >= 0.7.2.

Environment

tgaudillat02 commented 11 months ago

Hello, Any update ?