search

microsoft / CSharpClient-for-Kafka

.Net implementation of the Apache Kafka Protocol that provides basic functionality through Producer/Consumer classes. The project also offers balanced consumer implementation.
Other
463 stars 95 forks source link

ConsumerOffset fix when OffsetOutOfRange #61

Closed YixuanLiu closed 8 years ago

YixuanLiu commented 8 years ago

The fix is first, FetcherRunnable shouldn't just set the consumedoffset as it doesn't own how far it consumed. When FetcherRunnable detects about outofrange error, it signal PartitionTopicInfo to fix its offset. Creae a private variable lastKnownGoodNextRequestOffset for pti to save the old marker of NextRequestOffset When OutOfRange happens. In the next iteration of fetcher, use that to caculate the actual messages in the queue. After the successful fetch, it set back to align with NextRequestOffset.

Also cleaned up messageOffset as it is never being used.

msftclas commented 8 years ago

Hi @YixuanLiu, I'm your friendly neighborhood Microsoft Pull Request Bot (You can call me MSBOT). Thanks for your contribution!

It looks like you're a Microsoft contributor (Yixuan Liu). If you're full-time, we DON'T require a Contribution License Agreement. If you are a vendor, please DO sign the electronic Contribution License Agreement. It will take 2 minutes and there's no faxing! https://cla.microsoft.com.

TTYL, MSBOT;

soumyajit-sahu commented 8 years ago

LGTM