search

IBM / sarama

Sarama is a Go library for Apache Kafka.
MIT License
11.57k stars 1.76k forks source link

Metadata check returns nil leader #2473

Closed franmoore05 closed 1 year ago

franmoore05 commented 1 year ago
Versions
Sarama Kafka Go
1.38.1 2.8.1 1.19.0
Configuration

What configuration values are you using for Sarama and Kafka?

default
Logs

When filing an issue please provide logs from Sarama and Kafka if at all possible. You can set sarama.Logger to a log.Logger to capture Sarama debug output.

logs: CLICK ME

``` | [Sarama] 2023/05/15 09:15:13 client/metadata found some partitions to be leaderless [Sarama] 2023/05/15 09:15:14 client/metadata retrying after 250ms... (1 attempts remaining) [Sarama] 2023/05/15 09:15:14 client/metadata fetching metadata for [topic] from broker kafka-3:19092 [Sarama] 2023/05/15 09:15:14 client/metadata found some partitions to be leaderless kafka: error while consuming cantabular-dataset-instance-started/0: kafka server: In the middle of a leadership election, there is currently no leader for this partition and hence it is unavailable for writes [Sarama] 2023/05/15 09:15:14 consumergroup/session/sarama-48be8fd5-1874-4056-b59d-eb50b7733216/39 heartbeat loop stopped [Sarama] 2023/05/15 09:15:14 consumergroup/session/sarama-48be8fd5-1874-4056-b59d-eb50b7733216/39 released [Sarama] 2023/05/15 09:15:14 client/metadata fetching metadata for [topic] from broker kafka-3:19092 ```

Problem Description

If I stop the kafka broker which is the leader in my cluster, Sarama never seems to return the new leader from the metadata check even though a new leader has been selected by kafka. It is just stuck in an endless loop of the above log messages until the original leader is brought back online. Please can you advise. Thanks.

franmoore05 commented 1 year ago

No longer an issue, have discovered the source of the problem being replication factors not set correctly.