Periodic task name is missing in TaskResult

[Mapiarz]

Packages: celery 5.2.7, django-celery-beat 2.5.0, django-celery-results 2.5.0, django 3.2.18. Backed by postgres and rabbitmq.

Yes, I have set CELERY_RESULT_EXTENDED to True and other metadata is there, but not periodic task name. Not a duplicate of https://github.com/celery/django-celery-results/issues/342 - the periodic task name is missing in both list and detail view, in fact, it's not stored in the DB in the first place.

Any ideas?

[cveilleux]

I am seeing the same behavior after updating to 2.4.0 from 2.2.0, task names are missing from admin and database.

Thanks for pointing me to the undocumented (?) breaking change regarding result_extended, for me it was the solution.

I am setting the celery config directly, not using the CELERY_XXX django settings shim, although it should be equivalent.

app = Celery("myproj")
app.conf.update(
    [...]
    result_extended=True,
)

Double-check your CELERY_RESULT_EXTENDED is working and that the config does get sent all the way to celery.

[auvipy]

Thanks for pointing me to the undocumented (?) breaking change regarding result_extended, for me it was the solution.

I am setting the celery config directly, not using the CELERY_XXX django settings shim, although it should be equivalent.

Would you mind what is missing on maintainers side which will help to get rid of this undocumented breaking change? any suggestion or help would be highly appreciated

[cveilleux]

Thanks for asking and putting time to maintain this library.

The change was technically a fix, but I would argue that since it changes a core behavior of the library, it is also a breaking change.

Maybe add a note with a bold BREAKING prefix to the 2.4.0 changelog.

I did not intend to hijack @Mapiarz issue. He does mention he did set CELERY_RESULT_EXTENDED to true.

If the setting is indeed set properly and you are sure it gets passed down to the celery conf object, this could be a duplicate of https://github.com/celery/django-celery-results/issues/289

[andrei-v1]

Any ideas?

Are you using RabibitMQ or Redis as broker? I had same problem when I used RabibitMQ, when I switched to Redis everything works fine as expected

[dalmasonto]

Using redis solves the missing periodic_task_name

[Mapiarz]

@cveilleux It probably has to do with eager mode, yes.

Please stop suggesting I switch the broker. I'm already on RabbitMQ because I switched from Redis to work around other issues, e.g. https://github.com/celery/celery/issues/2481.

[dalmasonto]

Use this combination then, it might help

Django==4.0.4
celery==5.2.7
redis==4.5.5
django-celery-beat==2.5.0
django-celery-results==2.5.1

These are the versions am using in my project. They work well with me

[ribonred]

i use this version below and have same problem. indeed when change the broker into redis periodic_task_name appear.

i'm following until kombu.messaging.Producer._publish and end up in channel.basic_publish i dont have any idea to follow more.. where the properties periodic_task_name are disappear. and no matter what i put options in task.apply_async just not come up in consumer

PRODUCER / BEFORE SENT:

{
   "priority":"None",
   "content_type":"application/json",
   "content_encoding":"utf-8",
   "application_headers":{
      "lang":"py",
      "task":"echo_debug",
      "id":"225d853e-ddc4-4e74-b750-c0909389639f",
      "shadow":"None",
      "eta":"None",
      "expires":"None",
      "group":"None",
      "group_index":"None",
      "retries":0,
      "timelimit":[
         "None",
         "None"
      ],
      "root_id":"225d853e-ddc4-4e74-b750-c0909389639f",
      "parent_id":"None",
      "argsrepr":"()",
      "kwargsrepr":"{}",
      "origin":"gen428598@soberdev",
      "ignore_result":false
   },
   "correlation_id":"225d853e-ddc4-4e74-b750-c0909389639f",
   "reply_to":"35404841-dbc5-3e90-80aa-98307e5e99e7",
   "periodic_task_name":"something",
   "delivery_mode":2
}

CONSUMER:

{
   "content_type":"application/json",
   "content_encoding":"utf-8",
   "application_headers":{
      "lang":"py",
      "task":"echo_debug",
      "id":"225d853e-ddc4-4e74-b750-c0909389639f",
      "shadow":"None",
      "eta":"None",
      "expires":"None",
      "group":"None",
      "group_index":"None",
      "retries":0,
      "timelimit":[
         "None",
         "None"
      ],
      "root_id":"225d853e-ddc4-4e74-b750-c0909389639f",
      "parent_id":"None",
      "argsrepr":"()",
      "kwargsrepr":"{}",
      "origin":"gen451061@soberdev",
      "ignore_result":false
   },
   "delivery_mode":2,
   "correlation_id":"225d853e-ddc4-4e74-b750-c0909389639f",
   "reply_to":"35404841-dbc5-3e90-80aa-98307e5e99e7"
}

version:

django-celery-results = "^2.5.0"
Django = "^4.0.6"
celery==5.2.7
django-celery-beat==2.5.0

[nick-ge]

Is there something new to this issue?

[adampei]

I have the same problem, anyone solved this problem?

[jesusfer]

Setting CELERY_RESULT_EXTENDED to True solved this issue for me.

Django==4.2.6 celery==5.3.4 django-celery-beat==2.5.0 django_celery_results==2.5.1 Using Redis as broker.

[oatnog]

I'm seeing this with rabbitmq as broker. CELERY_RESULT_EXTENDED hasn't made a difference. Anything I can provide to help reproduce this bug?

[AngryUbuntuNerd]

I have the same issue: RabbitMQ messages do not contain the periodic_task_name, even if CELERY_RESULT_EXTENDED is set to True, and therefore workers can not store it in the DB either.

The underlying reason is that periodic_task_name is not in the list of allowed top-level-properties for AMQP (see amqp sourcecode) so it gets discarded when passed over to RabbitMQ.

A quick solution I found was storing it in a header instead, here a monkey-patched approach:

from django_celery_beat.apps import BeatConfig
from django_celery_results.apps import CeleryResultConfig
...

def monkey_patch_beat(original_ready: Callable) -> Callable:
    def patch(original_init: Callable) -> Callable:
        from django_celery_beat.schedulers import ModelEntry, PeriodicTask

        def init(self: ModelEntry, model: PeriodicTask, app: Optional[Celery] = None) -> None:
            original_init(self, model, app)
            self.options['headers']['periodic_task_name'] = model.name

        return init

    def ready(self: BeatConfig) -> None:
        from django_celery_beat.schedulers import ModelEntry
        original_ready(self)

        ModelEntry.__init__ = patch(ModelEntry.__init__)

    return ready

BeatConfig.ready = monkey_patch_beat(BeatConfig.ready)

def monkey_patch_results(original_ready: Callable) -> Callable:
    def patch(original_get_extended_properties: Callable) -> Callable:
        from django_celery_results.backends import DatabaseBackend

        def get_extended_properties(self: DatabaseBackend, request: Optional[Context], traceback: Optional) -> dict:
            extended_props = original_get_extended_properties(self, request, traceback)
            if not extended_props.get('periodic_task_name'):
                extended_props['periodic_task_name'] = getattr(request, 'periodic_task_name', None)
            return extended_props

        return get_extended_properties

    def ready(self: CeleryResultConfig) -> None:
        from django_celery_results.backends import DatabaseBackend
        original_ready(self)

        DatabaseBackend._get_extended_properties = patch(DatabaseBackend._get_extended_properties)

    return ready

CeleryResultConfig.ready = monkey_patch_results(CeleryResultConfig.ready)

[AngryUbuntuNerd]

Notice there is a fix in the making in https://github.com/celery/django-celery-beat/pull/812 and https://github.com/celery/django-celery-results/pull/445