This PR introduces a scheduler for audio threads: idle tasks are all run on a single background async task, while live audio threads are grouped up into full sync threads. Tasks are dynamically promoted and demoted based on whether they have any live audio tracks. This adds in some other niceties like preventing packet buffer reallocations, where possible. I intend to add reclamation of packet buffers, worker threads, and offloading expensive tasks to other threads.
This defaults to grouping 16 live calls per thread, mainly to prevent issues with users not compiling in release mode and/or using weaker machines. This can be configured during Config creation, and should be tuned for users at scale -- for instance, a passthrough-only workload can serve up to ~660 calls per thread on a Ryzen 5700X.
To put this in context, if all calls are live, this defaults to a 16x reduction in the amount of threads used. If only 30% of calls are live, the remainder all go in a single async task, giving an asymptotic 53x reduction of thread count.
TODO:
[x] Cleanup & document the draft implementation
[x] Fix Benchmarks to account for scheduler
[x] Integrate testing of scheduler behaviours into harness
[x] Periodically reclaim threads and packet blocks
This PR introduces a scheduler for audio threads: idle tasks are all run on a single background async task, while live audio threads are grouped up into full sync threads. Tasks are dynamically promoted and demoted based on whether they have any live audio tracks. This adds in some other niceties like preventing packet buffer reallocations, where possible. I intend to add reclamation of packet buffers, worker threads, and offloading expensive tasks to other threads.
This defaults to grouping 16 live calls per thread, mainly to prevent issues with users not compiling in release mode and/or using weaker machines. This can be configured during
Configcreation, and should be tuned for users at scale -- for instance, a passthrough-only workload can serve up to ~660 calls per thread on a Ryzen 5700X.To put this in context, if all calls are live, this defaults to a 16x reduction in the amount of threads used. If only 30% of calls are live, the remainder all go in a single async task, giving an asymptotic 53x reduction of thread count.
TODO: