Open persona94 opened 6 days ago
Is there any other information I can provide? @pranavrth
Can you also provide time in the logs? Better to show human readable logs. Your app logger is not logging time.
JoinGroupRequest
is taking around 42 seconds.
"RECV [my-kafka-consumer-ac#consumer-2] [thrd:GroupCoordinator]: GroupCoordinator/0: Received JoinGroupResponse (v5, 271 bytes, CorrId 3, rtt 42039.28ms)"}
How many consumers is this coordinator serving? Is there some issue in the coordinaor? I don't know what is happening at the coordinator side which is taking 42s to respond.
Can you also provide time in the logs? Better to show human readable logs. Your app logger is not logging time.
Sure. I had trimmed the logs because they were a lot, I've attached the unmodified file from this morning book.log
JoinGroupRequest
is taking around 42 seconds.
"RECV [my-kafka-consumer-ac#consumer-2] [thrd:GroupCoordinator]: GroupCoordinator/0: Received JoinGroupResponse (v5, 271 bytes, CorrId 3, rtt 42039.28ms)"}
How many consumers is this coordinator serving? Is there some issue in the coordinaor? I don't know what is happening at the coordinator side which is taking 42s to respond.
This coordinator serves 2 consumers. Each in it's own consumer group. I run this consumer on 3 separate clusters (ie.e all different independent kafka instances) and all of them seem to have the same issue. From what I can tell this is not an issue on the other consumer which uses a c++ driver. Also, if I use kafka-python as my kafka driver I don't see this delay, I connect instantly
Description
The consumer subscribes to a topic in the following fashion
Inside
start_consumer()
the main while loop looks likeI see the consumer takes 45s from "Waiting for message" to move to "Message received".
I turned up debug logs in kafka and saw this
How to reproduce
Checklist
Please provide the following information:
confluent_kafka.version()
andconfluent_kafka.libversion()
): v2.4.0'auto.offset.reset':'earliest', 'client.id': 'my-kafka-consumer-ac', 'fetch.min.bytes': '200000', 'metadata.max.age.ms': '1000',
'debug': '..'
as necessary)