Azure / azure-functions-eventhubs-extension

Event Hubs extension for Azure Functions
MIT License
20 stars 26 forks source link

Retry loop should cancel on lease lost #92

Closed paulbatum closed 2 years ago

paulbatum commented 3 years ago

When using a retry policy with an eventhub trigger, the retry loop should be cancelled if the lease is lost. If the retry loop is not cancelled, this will lead to duplicate message processing or potentially broken message ordering, where one machine is waiting to retry a message while another machine has already acquired the lease and started processing.

@alrod notes:

It looks like we do not pass the cancellation token properly from “EventHubListener” to “EventProcessor”, we just create it here.

mathewc commented 2 years ago

Seems like the underlying issue here is that the retry logic here doesn't fully respect the CancellationToken passed in. In the loop it should check for cancellation before beginning a new attempt, and it should also pass the token in on the Task.Delay. If all of this is done, when the caller requests cancellation (e.g. due to a lost lease) retries will stop.

alrod commented 2 years ago

Fixed in track2: https://github.com/Azure/azure-sdk-for-net/pull/27340