search

dehora / nakadi-java

🌀 Client library for the Nakadi Event Broker (examples: http://bit.ly/njc-examples, site: https://dehora.github.io/nakadi-java/)
MIT License
30 stars 19 forks source link

Stops stream processor after retries are exceeded. #285

Closed dehora closed 7 years ago

dehora commented 7 years ago

This signals that the number of retries for a processor have been met so that the processor can perform a shutdown and clean up its underlying resources via stopStreaming.

For #284.

dehora commented 7 years ago

It turns out retry limits weren't honored for stream consumption, just calls like checkpointing.

The patch was tested with a simple loop of 100 processors competing for 1 partition, each processor configured with 2 retry attempts.

This shows how threads aren't cleaned up from before the patch:

without

This is with the patch and shows threads being cleaned up, eventually leaving just one processor active:

with