I have an Azure function with an event hub trigger with cardinality set to 'many'. The average processing time for my function is about 10 milliseconds, but about 0.2% of function executions takes over 10 seconds due to a problem with an external dependency. Since the event hub trigger waits for the function execution to complete before sending another batch of messages, this effectively leads to all events on that partition being delayed for 10 seconds.
Of course you can't just keep pushing batches of events to the function as soon as they arrive at event hub, we need some kind of backpressure mechanism. I suggest for the trigger to allow a configurable number of batches to be processed by the function in parallel (edit: per partition). Perhaps the configuration property can be called maxParallelExecutions, that way it could also be used when _singleDispatch is set to true.
Some more context: I am working on a high volume low latency IoT solution where any single message isn't very valuable, but having latency on a large chunk of messages leads to significant problems. I have mitigated the issue somewhat by simply putting timeouts of 500ms on all my I/O calls, but that still isn't ideal. I also tried a solution where within my function I defined an internal queue and some workers that handle the events asynchronously, and then just complete the function when the queue size gets under some threshold. Azure Functions isn't made for scheduling asynchronous tasks within a function execution though. Any logs that are written after the function completes don't arrive in log analytics, and I am not sure what else breaks. Instead, this solution should be implemented in the trigger, hence this request.
I have an Azure function with an event hub trigger with cardinality set to 'many'. The average processing time for my function is about 10 milliseconds, but about 0.2% of function executions takes over 10 seconds due to a problem with an external dependency. Since the event hub trigger waits for the function execution to complete before sending another batch of messages, this effectively leads to all events on that partition being delayed for 10 seconds.
Of course you can't just keep pushing batches of events to the function as soon as they arrive at event hub, we need some kind of backpressure mechanism. I suggest for the trigger to allow a configurable number of batches to be processed by the function in parallel (edit: per partition). Perhaps the configuration property can be called maxParallelExecutions, that way it could also be used when _singleDispatch is set to true.
Some more context: I am working on a high volume low latency IoT solution where any single message isn't very valuable, but having latency on a large chunk of messages leads to significant problems. I have mitigated the issue somewhat by simply putting timeouts of 500ms on all my I/O calls, but that still isn't ideal. I also tried a solution where within my function I defined an internal queue and some workers that handle the events asynchronously, and then just complete the function when the queue size gets under some threshold. Azure Functions isn't made for scheduling asynchronous tasks within a function execution though. Any logs that are written after the function completes don't arrive in log analytics, and I am not sure what else breaks. Instead, this solution should be implemented in the trigger, hence this request.
Edit: I am looking for something similar to the maxConcurrency and ordering settings in the confluent parallel consumer: https://github.com/confluentinc/parallel-consumer#unordered.