search

tobymao / saq

Simple Async Queues
https://saq-py.readthedocs.io/en/latest/
MIT License
583 stars 41 forks source link

Option to restart worker based on metrics (i.e. memory consumption / tasks processed) #190

Closed provinzkraut closed 5 days ago

provinzkraut commented 5 days ago

It would be very convenient to have the option to restart a worker process once it hits a certain threshold, such as memory consumption or number of tasks processed. In long-running deployments, this can help with things such as memory leaks that are out of control of the application.

Celery allows this with their worker_max_tasks_per_child and worker_max_memory_per_child, which I default all my celery deployments to. Migrating to saq, this has been a bit of a holdup.

If this is something you'd consider adding, I'd be happy to contribute a PR :)

tobymao commented 5 days ago

sure, i'm open to it, feel free to make a PR. thank you.

closing as not planned since core team won't work on it. but will work with you to get it in.