job_id = kwargs.pop('wooey_job') keyerror: wooey_job

[carlsunderman]

have gotten this error with python 3.x and 2.x on a new install using the sample script on multiple machines.

[carlsunderman]

[2019-02-22 15:39:53,981: INFO/MainProcess] Received task: wooey.tasks.submit_script[be4c78dd-a754-40e1-a2a3-55ceace64d18] [2019-02-22 15:39:53,992: ERROR/MainProcess] Pool callback raised exception: KeyError('wooey_job',) Traceback (most recent call last): File "d:\programdata\anaconda3\envs\py36\lib\site-packages\django\db\models\query.py", line 464, in get_or_create return self.get(**lookup), False File "d:\programdata\anaconda3\envs\py36\lib\site-packages\django\db\models\query.py", line 380, in get self.model._meta.object_name django_celery_results.models.DoesNotExist: TaskResult matching query does not exist.

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "d:\programdata\anaconda3\envs\py36\lib\site-packages\billiard\pool.py", line 1747, in safe_apply_callback fun(args, kwargs) File "d:\programdata\anaconda3\envs\py36\lib\site-packages\celery\worker\request.py", line 367, in on_failure self.id, exc, request=self, store_result=self.store_errors, File "d:\programdata\anaconda3\envs\py36\lib\site-packages\celery\backends\base.py", line 157, in mark_as_failure traceback=traceback, request=request) File "d:\programdata\anaconda3\envs\py36\lib\site-packages\celery\backends\base.py", line 322, in store_result request=request, kwargs) File "d:\programdata\anaconda3\envs\py36\lib\site-packages\django_celery_results\backends\database.py", line 35, in _store_result task_kwargs=task_kwargs, File "d:\programdata\anaconda3\envs\py36\lib\site-packages\django_celery_results\managers.py", line 50, in _inner return fun(args, kwargs) File "d:\programdata\anaconda3\envs\py36\lib\site-packages\django_celery_results\managers.py", line 126, in store_result obj, created = self.get_or_create(task_id=task_id, defaults=fields) File "d:\programdata\anaconda3\envs\py36\lib\site-packages\django\db\models\manager.py", line 85, in manager_method return getattr(self.get_queryset(), name)(*args, *kwargs) File "d:\programdata\anaconda3\envs\py36\lib\site-packages\django\db\models\query.py", line 466, in get_or_create return self._create_object_from_params(lookup, params) File "d:\programdata\anaconda3\envs\py36\lib\site-packages\django\db\models\query.py", line 499, in _create_object_from_params params = {k: v() if callable(v) else v for k, v in params.items()} File "d:\programdata\anaconda3\envs\py36\lib\site-packages\django\db\models\query.py", line 499, in params = {k: v() if callable(v) else v for k, v in params.items()} File "d:\programdata\anaconda3\envs\py36\lib\site-packages\celery\app\trace.py", line 642, in __protected_call__ return orig(self, args, kwargs) File "d:\programdata\anaconda3\envs\py36\lib\site-packages\celery\app\task.py", line 375, in call return self.run(*args, **kwargs) File "d:\programdata\anaconda3\envs\py36\lib\site-packages\wooey\tasks.py", line 79, in submit_script job_id = kwargs.pop('wooey_job') KeyError: 'wooey_job'

[Chris7]

Could you paste the output of pip freeze and how you are setting up wooey?

[carlsunderman]

here is pip freeze from my python 3.6 env

https://justpaste.it/6aj8o

I set it up via the instructions. installed with pip then wooify -p myproject

i did notice that setting celery to false in settings, it worked, in both environments.

i launched celery per the upgrade instructions

celery -A myproject worker -l info

[Chris7]

It looks like the kombu filesystem broker on windows is at fault. I'll see if there's a way to salvage it at all. Otherwise, setting up a real broker will resolve this problem (like rabbit). You can see these settings in your_project/settings/user_settings.py with CELERY_BROKER_URL

[Chris7]

Ah, it's realted to this issue: https://github.com/celery/celery/issues/4081. Can you set FORKED_BY_MULTIPROCESSING=1 as an environment variable?

[Chris7]

Additionally, there is this bit about running on windows in the docs:

On Windows, the --beat option may not be supported and the eventlet pool will need to be specified. This looks like:
celery -A your_project_name worker --pool=eventlet -l info

I see you came to the new celery command through the upgrade docs, I'll have to update that to include this disclaimer as well.

[carlsunderman]

@Chris7 ,

yes, it seems to be working now. I launched celery as you said, using eventlet and modified wooey_celery_app.py, adding os.environ.setdefault('FORKED_BY_MULTIPROCESSING', '1') before importing celery from:

Ah, it's realted to this issue: celery/celery#4081. Can you set FORKED_BY_MULTIPROCESSING=1 as an environment variable?

thank you very much