celery / django-celery-results

Celery result back end with django
Other
684 stars 203 forks source link

Chords not supported when using 'django-db' backend #341

Open ShuangLau opened 2 years ago

ShuangLau commented 2 years ago

Similar to the issue mentioned here: https://github.com/celery/django-celery-results/issues/52#issuecomment-608443564. We have met the same error when trying to run chord tasks using django-db as the result backend:

NotImplementedError
Starting chords requires a result backend to be configured. Note that a group chained with a task is also upgraded to be a chord, as this pattern requires synchronization. Result backends that supports chords: Redis, Database, Memcached, and mor

Here is the lib versions:

Django==3.2.13
django-celery-results==2.3.1
django-health-check==3.16.5
django-celery-beat==2.3.0
psycopg2-binary==2.9.1
django-extensions==3.1.5
gunicorn==20.1.0
django-cors-headers==3.13.0
celery==5.2.3

The setting we configured:

CELERY_BROKER_URL = f'redis://{os.getenv("REDIS_HOST")}:{os.getenv("REDIS_PORT")}'
CELERY_RESULT_BACKEND = "django-db"
CELERY_CACHE_BACKEND = "django-cache"

Is it possible to use django-db with chord tasks or it is not supported yet? If it is supported, how should I fix this error? Thanks a lot.

ShuangLau commented 2 years ago

Update:

I noticed that ChordCounter is missing in the django admin console. Does this mean anything?

Screen Shot 2022-09-07 at 4 00 11 PM
namper commented 2 years ago

@ShuangLau Experience same problem.

From my analysis i found that chord fails when sub tasks results are too big. This can be checked by running

ChordCounter.objects.all().values()

We see existing counters not cleaned up and hence callback is not executed.

P.S ChordCounter doesn't have ModelAdmin and that's why it doesn't show, so that's normal.

auvipy commented 1 year ago

should that be shown in the admin?

namper commented 1 year ago

should that be shown in the admin?

No, because they are not persistent objects, as they are deleted after counter is zero

auvipy commented 1 year ago

then we probably should document this as intended behaviour? btw, any thought on improving the mechanism to handle bigger data?

namper commented 1 year ago

Agreed, we should document.

Not sure about solution.

You can have same problem with redis backend if result are too big, fetching them are very slow, and solution to that is to store result in storage and retrieve it from there. this approach can be used to side step limitation of database backend.