Change the task manager task claiming algorithm to use a _search to retrieve candidate tasks, a _mget to prune the docs whose version number doesn't match, and then a _bulk to claim the tasks. This will increase the background task capacity in Serverless.
Business Value
Increased background task capacity, reducing the COGS for running alerting rules and actions, and providing a lower MTTD/MTTR.
Definition of Done
[ ] Performance tests against Stateful Elasticsearch to ensure performance did not degrade significantly
[ ] Performance tests against Serverless Elasticsearch to measure performance
[ ] If performance is degraded, follow up-issues are created to address the bottlenecks
Part of https://github.com/elastic/kibana/issues/155770
Feature Description
Change the task manager task claiming algorithm to use a _search to retrieve candidate tasks, a _mget to prune the docs whose version number doesn't match, and then a _bulk to claim the tasks. This will increase the background task capacity in Serverless.
Business Value
Increased background task capacity, reducing the COGS for running alerting rules and actions, and providing a lower MTTD/MTTR.
Definition of Done