This is a rather strange one. I have a queue configured to be worked by 16 threads, and looking at htop it does appear there are that many threads working. But watching the backburner logs (and the general throughput) it's only working one job at a time. So if there's a pause in the handling of one job (say a long response from an external service) then all other jobs are held back until that completes.
I reconfigured the queue to use one worker and got the same throughput. I'll keep digging into it to see if I can determine if it's my app's usage of Backburner that's doing it or if it's Backburner itself.
This is a rather strange one. I have a queue configured to be worked by 16 threads, and looking at
htopit does appear there are that many threads working. But watching the backburner logs (and the general throughput) it's only working one job at a time. So if there's a pause in the handling of one job (say a long response from an external service) then all other jobs are held back until that completes.I reconfigured the queue to use one worker and got the same throughput. I'll keep digging into it to see if I can determine if it's my app's usage of Backburner that's doing it or if it's Backburner itself.