search

IBM / sarama

Sarama is a Go library for Apache Kafka.
MIT License
11.43k stars 1.75k forks source link

Support AdminClient connection to Controller listeners #2981

Open roarvroom opened 2 weeks ago

roarvroom commented 2 weeks ago

Description

Since KIP-919 (Kafka 3.7.0) it's possible to connect to Controller listeners and talk to them using AdminClient. This is not yet supported in Sarama, as creating cluster admin fails (probably) due to protocol mismatch.

Additional context

https://cwiki.apache.org/confluence/display/KAFKA/KIP-919%3A+Allow+AdminClient+to+Talk+Directly+with+the+KRaft+Controller+Quorum+and+add+Controller+Registration

[Sarama debug] 2024/09/10 17:26:19 Initializing new client
[Sarama] 2024/09/10 17:26:19 ClientID is the default of 'sarama', you should consider setting it to something application-specific.
[Sarama] 2024/09/10 17:26:19 ClientID is the default of 'sarama', you should consider setting it to something application-specific.
[Sarama debug] 2024/09/10 17:26:19 client/metadata fetching metadata for all topics from broker kafka1:19093
[Sarama debug] 2024/09/10 17:26:19 Connected to broker at kafka1:19093 (unregistered)
[Sarama] 2024/09/10 17:26:19 client/metadata got error from broker -1 while fetching metadata: EOF
[Sarama debug] 2024/09/10 17:26:19 Closed connection to broker kafka1:19093
[Sarama] 2024/09/10 17:26:19 client/metadata no available broker to send metadata request to
[Sarama] 2024/09/10 17:26:19 client/brokers resurrecting 1 dead seed brokers
[Sarama] 2024/09/10 17:26:19 client/metadata retrying after 250ms... (2 attempts remaining)
[Sarama] 2024/09/10 17:26:19 ClientID is the default of 'sarama', you should consider setting it to something application-specific.
[Sarama debug] 2024/09/10 17:26:19 client/metadata fetching metadata for all topics from broker kafka1:19093
[Sarama debug] 2024/09/10 17:26:19 Connected to broker at kafka1:19093 (unregistered)
[Sarama] 2024/09/10 17:26:19 client/metadata got error from broker -1 while fetching metadata: EOF
[Sarama debug] 2024/09/10 17:26:19 Closed connection to broker kafka1:19093
[Sarama] 2024/09/10 17:26:19 client/metadata no available broker to send metadata request to
[Sarama] 2024/09/10 17:26:19 client/brokers resurrecting 1 dead seed brokers
[Sarama] 2024/09/10 17:26:19 client/metadata retrying after 250ms... (1 attempts remaining)
[Sarama] 2024/09/10 17:26:19 ClientID is the default of 'sarama', you should consider setting it to something application-specific.
[Sarama debug] 2024/09/10 17:26:19 client/metadata fetching metadata for all topics from broker kafka1:19093
[Sarama debug] 2024/09/10 17:26:19 Connected to broker at kafka1:19093 (unregistered)
[Sarama] 2024/09/10 17:26:19 client/metadata got error from broker -1 while fetching metadata: EOF
[Sarama debug] 2024/09/10 17:26:19 Closed connection to broker kafka1:19093
[Sarama] 2024/09/10 17:26:19 client/metadata no available broker to send metadata request to
[Sarama] 2024/09/10 17:26:19 client/brokers resurrecting 1 dead seed brokers
[Sarama] 2024/09/10 17:26:19 client/metadata retrying after 250ms... (0 attempts remaining)
[Sarama] 2024/09/10 17:26:19 ClientID is the default of 'sarama', you should consider setting it to something application-specific.
[Sarama debug] 2024/09/10 17:26:19 client/metadata fetching metadata for all topics from broker kafka1:19093
[Sarama debug] 2024/09/10 17:26:19 Connected to broker at kafka1:19093 (unregistered)
[Sarama] 2024/09/10 17:26:19 client/metadata got error from broker -1 while fetching metadata: EOF
[Sarama debug] 2024/09/10 17:26:19 Closed connection to broker kafka1:19093
[Sarama] 2024/09/10 17:26:19 client/metadata no available broker to send metadata request to
[Sarama] 2024/09/10 17:26:19 client/brokers resurrecting 1 dead seed brokers
[Sarama debug] 2024/09/10 17:26:19 Closing Client
panic: kafka: client has run out of available brokers to talk to: EOF
nikolaydimitrov commented 1 week ago

+1

Also, until now I was using v1.38.1 and it was working. Now I am getting the same errors as above with v1.43.3