Our clients reported that utubettl:take() loads CPU too much and takes too much time when there is a utube with a big amount of tasks (+100.000).
We need a benchmark for the case in t/benchmark. It could be a queue with utubettl driver and 5-10 utubes with ~50.000 tasks + 5-10 parallel clients with take() + ack(). It will help to identify the problem.
We need an another benchmark in t/benchmark. With a big number of utubes (10.000) with a small number of tasks on each (4-5) + 5-10 parallel clients with take() + ack(). This way we can make sure that our fix doesn't degrade performance.
We need to fix the problem.
I suggest to make the task in the order:
Create a benchmark for the problem and ensure that we have the problem.
You offer solutions to the problem in the issue, which we will discuss.
Fix the problem after discussion. Ensure that it helps.
Create a regression benchmark and ensure that we don't degrade performance.
Our clients reported that
utubettl:take()loads CPU too much and takes too much time when there is autubewith a big amount of tasks (+100.000).utubettldriver and 5-10utubes with ~50.000 tasks + 5-10 parallel clients with take() + ack(). It will help to identify the problem.utubes (10.000) with a small number of tasks on each (4-5) + 5-10 parallel clients with take() + ack(). This way we can make sure that our fix doesn't degrade performance.I suggest to make the task in the order: