Open danielcohenlive opened 6 years ago
See the stacktrace below
11:40:18 celery_general.1 | [2018-10-18 15:40:18,210: ERROR/ForkPoolWorker-2] Task xxx.apps.marketo.tasks.update_marketingdata_lead.update_marketingdata_lead[9f1605d0-bfe2-4549-8cf2-91abcdc8fd7b] raised unexpected: AssertionError('daemonic processes are not allowed to have children',) 11:40:18 celery_general.1 | Traceback (most recent call last): 11:40:18 celery_general.1 | File "venv/lib/python2.7/site-packages/celery/app/trace.py", line 367, in trace_task 11:40:18 celery_general.1 | R = retval = fun(*args, **kwargs) 11:40:18 celery_general.1 | File "venv/lib/python2.7/site-packages/celery/app/trace.py", line 622, in __protected_call__ 11:40:18 celery_general.1 | return self.run(*args, **kwargs) 11:40:18 celery_general.1 | File "xxxx/apps/marketo/tasks/update_marketingdata_lead.py", line 81, in update_marketingdata_lead 11:40:18 celery_general.1 | _enqueue_lead_data(email, to_update, user_id) 11:40:18 celery_general.1 | File "xxxx/apps/marketo/tasks/update_marketingdata_lead.py", line 93, in _enqueue_lead_data 11:40:18 celery_general.1 | get_lead_producer().put(json.dumps(lead_data)) 11:40:18 celery_general.1 | File "xxxx/apps/xxxx_kinesis/__init__.py", line 39, in get_lead_producer 11:40:18 celery_general.1 | buffer_time=5 * 60, 11:40:18 celery_general.1 | File "venv/lib/python2.7/site-packages/kinesis/producer.py", line 142, in __init__ 11:40:18 celery_general.1 | max_size=max_size, boto3_session=boto3_session) 11:40:18 celery_general.1 | File "venv/lib/python2.7/site-packages/kinesis/producer.py", line 78, in __init__ 11:40:18 celery_general.1 | self.start() 11:40:18 celery_general.1 | File "venv/lib/python2.7/site-packages/offspring/process.py", line 55, in start 11:40:18 celery_general.1 | self.process.start() 11:40:18 celery_general.1 | File "/usr/local/Cellar/python@2/2.7.15_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/multiprocessing/process.py", line 124, in start 11:40:18 celery_general.1 | 'daemonic processes are not allowed to have children' 11:40:18 celery_general.1 | AssertionError: daemonic processes are not allowed to have children
My understanding is that using billiard instead of multiprocess will solve this issue
billiard
multiprocess
See the stacktrace below
My understanding is that using
billiard
instead ofmultiprocess
will solve this issue