AttributeError: 'Settings' object has no attribute 'ONCE'

[Dean-Christian-Armada]

I am always receiving this error in celery logs after execution like

[2017-07-10 15:40:33,053: CRITICAL/MainProcess] Task provider.KENO.tasks.add[1da681d2-2815-4257-b139-01facaaef67d] INTERNAL ERROR: AttributeError(u"'Settings' object has no attribute 'ONCE'",)
Traceback (most recent call last):
  File "/Users/deanchristianarmada/Desktop/projects/radars/radar/lib/python2.7/site-packages/celery/app/trace.py", line 299, in trace_task
    state, retval, uuid, args, kwargs, None,
  File "/Users/deanchristianarmada/Desktop/projects/radars/radar/lib/python2.7/site-packages/celery_once/tasks.py", line 129, in after_return
    self.once_backend.clear_lock(key)
  File "/Users/deanchristianarmada/Desktop/projects/radars/radar/lib/python2.7/site-packages/celery_once/tasks.py", line 52, in once_backend
    return import_backend(self.once_config)
  File "/Users/deanchristianarmada/Desktop/projects/radars/radar/lib/python2.7/site-packages/celery_once/tasks.py", line 48, in once_config
    return self.config.ONCE
  File "/Users/deanchristianarmada/Desktop/projects/radars/radar/lib/python2.7/site-packages/celery/datastructures.py", line 353, in __getattr__
    type(self).__name__, k))
AttributeError: 'Settings' object has no attribute 'ONCE'

My celery version is 3.1.24. I tried it in 4.02 and the error still exists

My code is:

from celery import Celery
from celery_once import QueueOnce

celery = Celery('tasks', broker='amqp://guest:guest@localhost:5672//')
celery.conf.ONCE = {
  'backend': 'celery_once.backends.Redis',
  'settings': {
    'url': 'redis://127.0.0.1:6379',
    'default_timeout': 60 * 60
  }
}

@celery.task(base=QueueOnce)
def add(x, y):
    return x + y

I run it through terminal celery -A proj -l info and coded in python manage.py shell like:

>>> from app import add
>>> x = add.apply_async(args=[6, 74])
>>> 

[cameronmaske]

Hey Dean-Christian-Armada thanks for the bug report. I've been trying to re-create this on my side following your instructions but haven't been able to.

The code example you provided, is it in one file? One thing to check is that you are importing the declared celery and not directly via import celery.

[Dean-Christian-Armada]

yes, only in one file app/prov/KEN/tasks.py

I'm using django by the way

settings.py

APP_TASKS = [
    'prov.KEN',
]
INSTALLED_APPS += APP_TASKS

Then I have a celery.py in the same folder as my settings.py the code is:

from __future__ import absolute_import
import os
from celery import Celery
from django.conf import settings

# set the default Django settings module for the 'celery' program.
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'radar.settings')
app = Celery('app')

# Using a string here means the worker will not have to
# pickle the object when using Windows.
app.config_from_object('django.conf:settings')
app.autodiscover_tasks(lambda: settings.INSTALLED_APPS)

[cameronmaske]

From that settings.py and app.config_from_object('django.conf:settings') you posted above, it seems to be missing any ONCE configuration options.

In your settings.py, could you try adding

ONCE = {
  'backend': 'celery_once.backends.Redis',
  'settings': {
    'url': 'redis://127.0.0.1:6379',
    'default_timeout': 60 * 60
  }
}

or

CELERY_ONCE = {
  'backend': 'celery_once.backends.Redis',
  'settings': {
    'url': 'redis://127.0.0.1:6379',
    'default_timeout': 60 * 60
  }
}

And seeing if that works?

[codekoala]

I encountered the same problem in one of my projects using Celery 4.1. I can confirm that adding the first block to my celery configuration resolved the problem.

Would it make sense to set some "sane defaults" for celery_once's config?

[cameronmaske]

@codekoala For another django project? If so, it's maybe worth mentioning in the docs someone. I'm against adding defaults, I like following Python's mantra of "Explicit is better than implicit."

[codekoala]

Fair enough.

My project does not use Django at all. It's just standard Python.

[evetion]

For those that still encounter the same issue in using Django and celery_once, here's how I've set this up:

celery.py

...
app = Celery('yourapp')

app.conf.ONCE = settings.CELERY_ONCE  # force CELERY_ONCE to load settings
app.config_from_object('django.conf:settings')
app.autodiscover_tasks(lambda: settings.INSTALLED_APPS)
...

settings.py

...
CELERY_ONCE = {
      'backend': 'celery_once.backends.Redis',
      'settings': {
        'url': 'redis://127.0.0.1:6379/0',
        'default_timeout': 60 * 60
      }
    }
...

For those that also run Django tests on their tasks that use QueueOnce, here's how I mock the redis backend.

test_something.py

....
from django.test import TestCase
from fakeredis import FakeStrictRedis
from mock import patch

...

class AsyncTaskTest(TestCase):

    def setUp(self):
        self.get_redis = patch('celery_once.backends.redis.get_redis')
        self.mocked_redis = self.get_redis.start()

        self.redis = FakeStrictRedis()
        self.mocked_redis.return_value = self.redis

    @patch('yourapp.tasks.call_command')
    def test_your_task(self, mockCall):
        """
        Assert that your_task
        calls your_command only once.
        """
        your_task.delay()

        # Set redis key with TTL 100 seconds from now
        # so subsequent tasks won't run
        self.redis.set('qo_your_app.tasks.your_task', int(time()) + 100)

        your_task.delay()
        your_task.delay()

        mockCall.assert_called_with('your_command')
        self.assertEqual(mockCall.call_count, 1)

    def tearDown(self):
        self.get_redis.stop()