search

ah- / rdkafka-dotnet

C# Apache Kafka client
Other
242 stars 73 forks source link

Kafka 0.10 Consumer - Group is dead and removed #56

Closed ferodss closed 8 years ago

ferodss commented 8 years ago

I'm using Kafka 0.10 and trying to create a consumer app but when I run the app my consumer is disconected from broker. When I run the console consumer that came with kafka it works.

On Kafka config I have advertised.listeners=PLAINTEXT://localhost:9092

Seems like the consumer is receiving wrong offset:

7|2016-08-16 17:29:49.880|rdkafka#consumer-1|FETCHSTART| List with 2 partition(s):
7|2016-08-16 17:29:49.880|rdkafka#consumer-1|FETCHSTART|  test010 [0] offset INVALID
7|2016-08-16 17:29:49.880|rdkafka#consumer-1|FETCHSTART|  test010 [1] offset INVALID

On this gist you can see the full consumer log and the code.

Kafka log

[2016-08-16 17:25:40,349] INFO [GroupCoordinator 0]: Preparing to restabilize group consumer-example-01 with old generation 0 (kafka.coordinator.GroupCoordinator)
[2016-08-16 17:25:40,358] INFO [GroupCoordinator 0]: Stabilized group consumer-example-01 generation 1 (kafka.coordinator.GroupCoordinator)
[2016-08-16 17:25:40,575] INFO [GroupCoordinator 0]: Assignment received from leader for group consumer-example-01 for generation 1 (kafka.coordinator.GroupCoordinator)
[2016-08-16 17:25:41,745] INFO [GroupCoordinator 0]: Preparing to restabilize group consumer-example-01 with old generation 1 (kafka.coordinator.GroupCoordinator)
[2016-08-16 17:25:41,748] INFO [GroupCoordinator 0]: Group consumer-example-01 generation 1 is dead and removed (kafka.coordinator.GroupCoordinator)

Am I missing something?

ah- commented 8 years ago

Hi, thanks for posting the full gist, that's very useful for debugging!

This all looks good, I think it's just that the Console.ReadLine() to keep it alive is a bit too late. By the point where it reaches the ReadLine it has already shut down the consumer because it exited the using (var consumer ...) {} scope.

Try putting the ReadLine directly after Console.WriteLine($"Subscribed to: [{string.Join(", ", consumer.Subscription)}]");.

ferodss commented 8 years ago

It works! My bad.

Thanks man!