After trying different alternative approaches, such as rate of change between the last 2 requests, or rate of change between the last query and the worst performing query, I settled on the current approach which works and is simple, with just an added constrain to the maximum group size, which now cannot be longer than a % of the remaining shards.
Before:
After:
Additionally:
Fast responses less aggressively increase the group size (50% or 10%)
Check for the validity of the selector before passing it to the language provider to prevent flashing errors
After trying different alternative approaches, such as rate of change between the last 2 requests, or rate of change between the last query and the worst performing query, I settled on the current approach which works and is simple, with just an added constrain to the maximum group size, which now cannot be longer than a % of the remaining shards.
Before:
After:
Additionally: