Closed Dean-Christian-Armada closed 7 years ago
Hey Dean-Christian-Armada thanks for the bug report. I've been trying to re-create this on my side following your instructions but haven't been able to.
The code example you provided, is it in one file?
One thing to check is that you are importing the declared celery
and not directly via import celery
.
yes, only in one file app/prov/KEN/tasks.py
I'm using django by the way
settings.py
APP_TASKS = [
'prov.KEN',
]
INSTALLED_APPS += APP_TASKS
Then I have a celery.py in the same folder as my settings.py the code is:
from __future__ import absolute_import
import os
from celery import Celery
from django.conf import settings
# set the default Django settings module for the 'celery' program.
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'radar.settings')
app = Celery('app')
# Using a string here means the worker will not have to
# pickle the object when using Windows.
app.config_from_object('django.conf:settings')
app.autodiscover_tasks(lambda: settings.INSTALLED_APPS)
From that settings.py
and app.config_from_object('django.conf:settings')
you posted above, it seems to be missing any ONCE
configuration options.
In your settings.py
, could you try adding
ONCE = {
'backend': 'celery_once.backends.Redis',
'settings': {
'url': 'redis://127.0.0.1:6379',
'default_timeout': 60 * 60
}
}
or
CELERY_ONCE = {
'backend': 'celery_once.backends.Redis',
'settings': {
'url': 'redis://127.0.0.1:6379',
'default_timeout': 60 * 60
}
}
And seeing if that works?
I encountered the same problem in one of my projects using Celery 4.1. I can confirm that adding the first block to my celery configuration resolved the problem.
Would it make sense to set some "sane defaults" for celery_once
's config?
@codekoala For another django project? If so, it's maybe worth mentioning in the docs someone. I'm against adding defaults, I like following Python's mantra of "Explicit is better than implicit."
Fair enough.
My project does not use Django at all. It's just standard Python.
For those that still encounter the same issue in using Django and celery_once, here's how I've set this up:
celery.py
...
app = Celery('yourapp')
app.conf.ONCE = settings.CELERY_ONCE # force CELERY_ONCE to load settings
app.config_from_object('django.conf:settings')
app.autodiscover_tasks(lambda: settings.INSTALLED_APPS)
...
settings.py
...
CELERY_ONCE = {
'backend': 'celery_once.backends.Redis',
'settings': {
'url': 'redis://127.0.0.1:6379/0',
'default_timeout': 60 * 60
}
}
...
For those that also run Django tests on their tasks that use QueueOnce, here's how I mock the redis backend.
test_something.py
....
from django.test import TestCase
from fakeredis import FakeStrictRedis
from mock import patch
...
class AsyncTaskTest(TestCase):
def setUp(self):
self.get_redis = patch('celery_once.backends.redis.get_redis')
self.mocked_redis = self.get_redis.start()
self.redis = FakeStrictRedis()
self.mocked_redis.return_value = self.redis
@patch('yourapp.tasks.call_command')
def test_your_task(self, mockCall):
"""
Assert that your_task
calls your_command only once.
"""
your_task.delay()
# Set redis key with TTL 100 seconds from now
# so subsequent tasks won't run
self.redis.set('qo_your_app.tasks.your_task', int(time()) + 100)
your_task.delay()
your_task.delay()
mockCall.assert_called_with('your_command')
self.assertEqual(mockCall.call_count, 1)
def tearDown(self):
self.get_redis.stop()
I am always receiving this error in celery logs after execution like
My celery version is 3.1.24. I tried it in 4.02 and the error still exists
My code is:
I run it through terminal
celery -A proj -l info
and coded inpython manage.py shell
like: