Azure / azure-functions-eventhubs-extension

Event Hubs extension for Azure Functions
MIT License
20 stars 26 forks source link

Scale monitor does not respect initial offset position #96

Closed nilsgaeq closed 1 month ago

nilsgaeq commented 2 years ago

When resetting the checkpoints for a consumer, which occasionally could happen, the scale monitor assumes that the event count is last enqueued sequence number + 1.

https://github.com/Azure/azure-functions-eventhubs-extension/blob/5eb8543e47b448eb3b485a571a2ceb1ab404650f/src/Microsoft.Azure.WebJobs.Extensions.EventHubs/Listeners/EventHubsScaleMonitor.cs#L255

This could lead to the scale metrics reporting very high event count until the consumer has created new checkpoints for all the partitions. When encountering this situation, the scale monitor should respect the initial offset position, or just return a number that ensures that the vote is None until the consumer has checkpointed.