consumer: First message is lost if `auto.offset.reset = latest`

skroutz / rafka

Kafka proxy with a simple API, speaking the Redis protocol

https://engineering.skroutz.gr/blog/kafka-rails-integration/

GNU General Public License v3.0

8 stars 0 forks source link

consumer: First message is lost if `auto.offset.reset = latest` #52

Closed agis closed 6 years ago

agis commented 6 years ago

This is because we don't wait for the topic to be assigned to the consumer before calling Poll().

agis commented 6 years ago

When set on "largest" we can't avoid loosing messages that were produced before the consumer is assigned to a partition. This is known behavior and is expected when using "latest".