search

bsm / sarama-cluster

Cluster extensions for Sarama, the Go client library for Apache Kafka 0.9 [DEPRECATED]
MIT License
1.01k stars 222 forks source link

#Support : Way to get batch messages and commit if the batch is successful #274

Closed sanchitlohia-ovo closed 6 years ago

sanchitlohia-ovo commented 6 years ago

There doesn't seems a way to implement the below flow

  1. Read batch messages from Kafka.
  2. Process all the messages.
  3. If all messages processing succesful - Commit the offset
  4. If any of the message processing fails - Do not commit the offset and read the same batch again and try to process the messages and if successful commit the offset.
dim commented 6 years ago

Unfortunately, there isn't, at least not easily. That's why I have put a lot of effort into https://github.com/Shopify/sarama/pull/1099 :) This repo is going to be deprecated in favour of the latest sarama release.