microsoft / durabletask-mssql

Microsoft SQL storage provider for Durable Functions and the Durable Task Framework
MIT License
87 stars 32 forks source link

Support scale-out to 200 replicas #58

Open cgillum opened 2 years ago

cgillum commented 2 years ago

The MSSQL backend currently uses constant-rate polling to find new work to execute. As the number of replicas running this code increases, the load on the MSSQL database increases with it to the point where adding new replicas can reduce overall throughput.

This item tracks doing whatever work is necessary (like dynamically changing polling frequency, etc.) to allow scale-out to as many as 200 replicas sharing a single, fixed-size database while still increasing throughput. The 200 number was chosen to match the Azure Functions maximum replica scale limit.