Currently watermarkPersistence.set is called before the poll handler is invoked.
This means that if the poll handler throws, the new watermark has already been stored, and could result in missed data on recovery.
This PR adjusts where the poll handler is called.
An added benefit of this change is that you can reliably use the watermark as a consistency guarantee, ensuring that all data has been successfully processed up to that round.
Currently
watermarkPersistence.set
is called before thepoll
handler is invoked. This means that if thepoll
handler throws, the new watermark has already been stored, and could result in missed data on recovery.This PR adjusts where the
poll
handler is called.An added benefit of this change is that you can reliably use the watermark as a consistency guarantee, ensuring that all data has been successfully processed up to that round.