Closed nodomain closed 3 years ago
The workers were not scaling or the actual cron ?
The cron itself doesn't have scaling as there was a problem in the past for running more than one cron.
Might not be an issue now though, so I'll just add it
Oh, no, you're right. The actual workers don't have scaling. Seems I just missed that. Sorry. Adding it now.
Released under 1.7.0
After some debugging my issue with Sentry being stuck processing events from a high traffic site where we use it for CSP reports and scaling Kafka, RDS and Redis during the process I found that the single worker was using 100% CPU. The template is not using autoscaling currently. What is the reason for this? I propose the same scaling rules like for all other tasks. I am happy to send a pull request if you agree. In my scenario I now temporarily scaled to 50 workers to let the queue be processed.