search

zendesk / racecar

Racecar: a simple framework for Kafka consumers in Ruby
Apache License 2.0
490 stars 94 forks source link

Consumer reset not needed on a :not_coordinator according to librdkafka #301

Open mensfeld opened 2 years ago

mensfeld commented 2 years ago

This code:

https://github.com/zendesk/racecar/blob/master/lib/racecar/consumer_set.rb#L165

handles the not_coordinator by resetting the consumer.

Based on this comment:

https://github.com/edenhill/librdkafka/issues/2630#issuecomment-598193001

Huhm, it seems like the coordinator is not re-queried when we get a NOT_COORD_FOR_GROUP error for OffsetCommitRequests, but it will be re-queried when we get the same error for HeartbeatRequests so the situation should fix itself within the heartbeat interval (by default within a couple of seconds).

Here it's about the not coord for group but the same principle of metadata state refresh applies to not_coord.

The consumer should reset itself without a need for a new instance.

Do you want me to fix this?

dasch commented 2 years ago

:+1: