rails / mission_control-jobs

Dashboard and Active Job extensions to operate and troubleshoot background jobs
MIT License
611 stars 71 forks source link

Any way to check number of idle workers or number of workers consuming a job? #189

Closed salmonsteak1 closed 2 weeks ago

salmonsteak1 commented 2 weeks ago

As per the title, are there any console helpers or any other way we can check to see the number of workers that are idle or consuming jobs? I'm looking for an accurate way to scale my solid queue instances, and it seems like CPU isn't the best way to do so since workers can be hogged up by jobs which are not CPU-intensive

Thank you!

rosa commented 2 weeks ago

For number of workers consuming a job at a given moment, you could check solid_queue_claimed_executions and the number of distinct process_id there. That should give you an accurate number unless some of the workers have died unceremoniously and there are still jobs pending to be released for them. You'd do this like:

SolidQueue::ClaimedExecution.distinct(:process_id).count
salmonsteak1 commented 2 weeks ago

Thanks Rosa!