search

oleksiyk / kafka

Apache Kafka 0.9 client for Node
MIT License
297 stars 85 forks source link

For enable_auto_commit False, no automatic re-join ; Error Heartbeat: local member_id was not recognized; this consumer needs to re-join #229

Closed silwalprabin closed 6 years ago

silwalprabin commented 6 years ago

I create consumer as: consumer = KafkaConsumer(group_id="my_group_id", bootstrap_servers="192.168.10.45:9092", auto_offset_reset='latest', enable_auto_commit=False, consumer_timeout_ms=1000) consumer.subscribe("myTopic")

After some days, I am getting this error: 2018-07-11_05:04:07.75431 WARNING: Marking the coordinator dead (node 0) for group my_group_id: Heartbeat session expired. 2018-07-11_05:04:10.87000 WARNING: Heartbeat: local member_id was not recognized; this consumer needs to re-join

Above error continues to repeat until I stop and run code again.

What I figure-out: I see code to rejoin while polling is done (with enable_auto_commit set to True) in line 250 of https://github.com/dpkp/kafka-python/blob/bc4cc434cddf403a35d0393d68ecfdbfad17c8e5/kafka/coordinator/consumer.py (function named poll(self)) , but I cannot see code to re-join when enable_auto_commit is set to False.

oleksiyk commented 6 years ago

Are you sure you submitted this to correct repo?

silwalprabin commented 6 years ago

Yes you are right. Sorry, I was supposed to put it under https://github.com/dpkp/kafka-python/ I am closing it now :)