search

Code4SA / aleph

Sift through large sets of structured and unstructured data, and find the people and companies you look for.
https://data.occrp.org
0 stars 6 forks source link

Make flask app context associate with new celery workers #22

Closed longhotsummer closed 7 years ago

longhotsummer commented 7 years ago

Was getting 'application not registered on db instance and no application bound to current context' when celery workers were restarted because of CELERYD_MAX_TASKS_PER_CHILD or when killed by the kernel to free memory. CELERYD_MAX_TASKS_PER_CHILD is needed to keep memory use under control http://chase-seibert.github.io/blog/2013/08/03/diagnosing-memory-leaks-python.html

This sets a conservative default of CELERYD_MAX_TASKS_PER_CHILD and makes sure a new app context is given to a new worker.