Open rgossiaux opened 5 months ago
Our Celery setup right now has beat running with every celery worker. During MH this year there were moments where the worker got backed up and we scaled up the # of workers, but when we did that, we noticed that Beat tasks were being duplicated.
Celery doesn't support this out of the box. We're using django-celery-beat and there might be some way to avoid this using that. There are also projects like https://github.com/sibson/redbeat and https://github.com/ybrs/single-beat
Our Celery setup right now has beat running with every celery worker. During MH this year there were moments where the worker got backed up and we scaled up the # of workers, but when we did that, we noticed that Beat tasks were being duplicated.
Celery doesn't support this out of the box. We're using django-celery-beat and there might be some way to avoid this using that. There are also projects like https://github.com/sibson/redbeat and https://github.com/ybrs/single-beat