search

elastic / kibana

Your window into the Elastic Stack
https://www.elastic.co/products/kibana
Other
19.47k stars 8.04k forks source link

[ResponseOps][Task Manager][mget Claimer] fix problem with limited concurrency starvation of unlimited concurrency tasks #184937

Open pmuellr opened 3 weeks ago

pmuellr commented 3 weeks ago

In PR implement task claiming strategy mget #180485 we implemented an alternative task claiming strategy, but it has the following problem:

If the number of limited concurrency tasks to run next exceeds the count of the tasks we do the initial search for, no tasks with unlimited concurrency will be returned for this polling cycle, as they would be "starved" by the limited concurrency tasks.

This is because we are searching all task types in one search, then applying limited concurrency on the results. We need to change this to do multiple searches - the unlimited concurrency search should not search on any limited concurrency tasks, and vice versa. Presumably we'd be doing one search for unlimited concurrency, and an additional search for each task type with limited concurrency.

I think we can do this with an _msearch. I'm thinking we put all the common matching clauses we need into the top-level filter of the query, in hopes that they make use of the potential filter caching. Then the query can provide the specific matching clauses for that sub-search.

elasticmachine commented 3 weeks ago

Pinging @elastic/response-ops (Team:ResponseOps)