Problem with longterm_scheduler_backend config

pythoneast commented 4 years ago

I have a trouble with using your library. I've added config for celery as mentioned in documentation:

app.conf.longterm_scheduler_backend = 'redis://redis:6739/1'

But got an error:

Traceback (most recent call last):
  File "/usr/local/bin/celery", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.8/site-packages/celery/__main__.py", line 16, in main
    _main()
  File "/usr/local/lib/python3.8/site-packages/celery/bin/celery.py", line 322, in main
    cmd.execute_from_commandline(argv)
  File "/usr/local/lib/python3.8/site-packages/celery/bin/celery.py", line 495, in execute_from_commandline
    super(CeleryCommand, self).execute_from_commandline(argv)))
  File "/usr/local/lib/python3.8/site-packages/celery/bin/base.py", line 305, in execute_from_commandline
    return self.handle_argv(self.prog_name, argv[1:])
  File "/usr/local/lib/python3.8/site-packages/celery/bin/celery.py", line 487, in handle_argv
    return self.execute(command, argv)
  File "/usr/local/lib/python3.8/site-packages/celery/bin/celery.py", line 415, in execute
    return cls(
  File "/usr/local/lib/python3.8/site-packages/celery/bin/base.py", line 308, in run_from_argv
    return self.handle_argv(prog_name,
  File "/usr/local/lib/python3.8/site-packages/celery/bin/base.py", line 393, in handle_argv
    return self(*args, **options)
  File "/usr/local/lib/python3.8/site-packages/celery/bin/base.py", line 253, in __call__
    ret = self.run(*args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/celery_longterm_scheduler/scheduler.py", line 113, in run
    get_scheduler(self.app).execute_pending(timestamp)
  File "/usr/local/lib/python3.8/site-packages/celery_longterm_scheduler/scheduler.py", line 36, in from_app
    return cls(app)
  File "/usr/local/lib/python3.8/site-packages/celery_longterm_scheduler/scheduler.py", line 31, in __init__
    app.conf['longterm_scheduler_backend'], app)
  File "/usr/local/lib/python3.8/site-packages/celery/utils/collections.py", line 449, in __getitem__
    return self.__missing__(key)
  File "/usr/local/lib/python3.8/site-packages/celery/utils/collections.py", line 276, in __missing__
    raise KeyError(key)
KeyError: 'longterm_scheduler_backend'

What am I doing wrong? Thank you in advance

jheld commented 4 years ago

Can you provide more system and configuration details?

jeffreybrowning commented 4 years ago

This is an issue I hit with django. For celery to work off of django's settings files, celery conf variables must be defined starting with CELERY_ and in all caps (or namespace arg should be edited). I believe Celery also knows what conf variables to expect and filters the longterm_scheduler_backend variable out. This means we need to explicitly set the conf variable in the django apps celery file:

# celery.py
from django.conf import settings

...

app.config_from_object('django.conf:settings', namespace='CELERY')
app.conf.longterm_scheduler_backend = settings.CELERY_LONGTERM_SCHEDULER_BACKEND

When calling the command for the cronjob, you need to pass arguments for the django app, just as you would normally when invoking celery.

celery -A <django_app_name> longterm_scheduler

ZeitOnline / celery_longterm_scheduler

Problem with longterm_scheduler_backend config #2