search

mtesseract / nakadi-client

Haskell Client Library for the Nakadi Event Broker
Other
13 stars 9 forks source link

Smart / Fast Cursor Committing #66

Closed mtesseract closed 6 years ago

mtesseract commented 6 years ago

Currently, when consuming a subscription, every batch cursor is committed individually. This is not required and even more: For increasing throughput this is not even desirable. Better would be a smart cursor committing logic which skips some cursors, depending on event throughput and timings.

Such a "smart" subscription sink would probably have to be implemented as some seperate thread.

mtesseract commented 6 years ago

Also, cursors on seperate partitions should be committed seperately. Furthermore, it looks like cursor commit failures should be discarded silently.