celery / django-celery-results

Celery result back end with django
Other
697 stars 206 forks source link

AppRegistryNotReady: Apps aren't loaded yet on calling a task #439

Open GitRon opened 2 months ago

GitRon commented 2 months ago

Hi there!

Bug description

I just set up this backend for the first time and thought I made a mistake, because everytime I call a task via celery with

celery -A apps.config.celery_settings call myproject.tasks.my_task

I get this error:

File "/usr/local/lib/python3.12/site-packages/django_celery_results/backends/init.py", line 2, in from .database import DatabaseBackend File "/usr/local/lib/python3.12/site-packages/django_celery_results/backends/database.py", line 14, in from ..models import ChordCounter File "/usr/local/lib/python3.12/site-packages/django_celery_results/models.py", line 18, in class TaskResult(models.Model): File "/usr/local/lib/python3.12/site-packages/django/db/models/base.py", line 129, in new app_config = apps.get_containing_app_config(module) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/django/apps/registry.py", line 260, in get_containing_app_config self.check_apps_ready() File "/usr/local/lib/python3.12/site-packages/django/apps/registry.py", line 138, in check_apps_ready raise AppRegistryNotReady("Apps aren't loaded yet.") django.core.exceptions.AppRegistryNotReady: Apps aren't loaded yet.

After about 2 hours I tried to start the task programmatically via

python manage.py shell
from myproject.tasks import my_task
my_task.delay()

This works and the result gets stored in the backend.

Approaches

I tried this and this an this was there in the first place.

Celery settings

import os
from pathlib import Path
from typing import Final

import django
from celery import Celery
from celery.signals import beat_init, worker_ready, worker_shutdown
from django.conf import settings

from apps.config.celery.bootstraps import LivenessProbe

READINESS_FILE: Final[Path] = Path("/tmp/celery_ready")

# set the default Django settings module for the 'celery' program.
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "apps.config.settings")

# Setup celery
celery_app = Celery("portal", broker=settings.CELERY_BROKER_URL)

# Add liveness probe for k8s
celery_app.steps["worker"].add(LivenessProbe)

# Using a string here means the worker doesn't have to serialize
# the configuration object to child processes.
# - namespace='CELERY' means all celery-related configuration keys
#   should have a `CELERY_` prefix.
celery_app.config_from_object("django.conf:settings", namespace="CELERY")

# Load task modules from all registered Django app configs.
celery_app.autodiscover_tasks()

# This will make sure the app is always imported when Django starts so that the celery decorator will use this app.
__all__ = ("celery_app", settings.CELERY_RETRY_DELAY, settings.CELERY_RETRY_MAX_TIMES)  # noqa: PLE0604

@beat_init.connect
def beat_ready(**_):
    READINESS_FILE.touch()

@worker_ready.connect
def worker_ready(**_):
    READINESS_FILE.touch()

@worker_shutdown.connect
def worker_shutdown(**_):
    READINESS_FILE.unlink(missing_ok=True)

I'm opening this issue because I think it seems to be a bug, right?