search

celery / celery

Distributed Task Queue (development branch)
https://docs.celeryq.dev
Other
24.8k stars 4.67k forks source link

Celery workers display error periodically if you set event_serializer to 'msgpack' #8285

Open MemoryLeak55 opened 1 year ago

MemoryLeak55 commented 1 year ago

Checklist

Mandatory Debugging Information

Optional Debugging Information

Related Issues and Possible Duplicates

Related Issues

Possible Duplicates

Environment & Settings

Celery version: 5.2.7 (dawn-chorus)

celery report Output:

``` accept_content: ['msgpack'] event_serializer: 'msgpack' result_accept_content: ['msgpack'] result_serializer: 'msgpack' task_serializer: 'msgpack' deprecated_settings: None software -> celery:5.2.7 (dawn-chorus) kombu:5.2.4 py:3.10.6 billiard:3.6.4.0 py-amqp:5.1.1 platform -> system:Linux arch:64bit, ELF kernel version:5.19.0-42-generic imp:CPython loader -> celery.loaders.app.AppLoader settings -> transport:amqp results:disabled accept_content: ['msgpack'] event_serializer: 'msgpack' result_accept_content: ['msgpack'] result_serializer: 'msgpack' task_serializer: 'msgpack' deprecated_settings: None ```

Steps to Reproduce

  1. Use defaults, rabbitmq broker.
  2. Set event_serializer = 'msgpack' and run more than 1 celery worker.
  3. Look at celery worker logs to see an error every 2 seconds

    Required Dependencies

    • Minimal Python Version: 3.10.6
    • Minimal Celery Version: 5.2.7
    • Minimal Kombu Version: 5.2.4
    • Minimal Broker Version: rabbitmq 3.9.13
    • Minimal Result Backend Version: rabbitmq 3.9.13
    • Minimal OS and/or Kernel Version: ubuntu 22.04 5.19.0-42-generic linux
    • Minimal Broker Client Version: N/A or Unknown
    • Minimal Result Backend Client Version: N/A or Unknown

Python Packages

pip freeze Output:

``` amqp==5.1.1 billiard==3.6.4.0 celery==5.2.7 click==8.1.3 click-didyoumean==0.3.0 click-plugins==1.1.1 click-repl==0.2.0 kombu==5.2.4 msgpack==1.0.5 prompt-toolkit==3.0.38 pytz==2023.3 six==1.16.0 vine==5.0.0 wcwidth==0.2.6 ```

Other Dependencies

N/A

Minimally Reproducible Test Case

celeryconfig.py ```python from celery import Celery class Config: event_serializer = 'msgpack' task_serializer = "msgpack" result_serializer = "msgpack" accept_content = ["msgpack"] result_accept_content = ["msgpack"] celery_app = Celery('tasks') celery_app.config_from_object(Config) ``` tasks.py (not relevant but included) ```python from celeryconfig import celery_app @celery_app.task() def add(x, y): return x + y ``` Now, run two workers worker1 ```shell celery -A tasks worker --loglevel=info --concurrency=1 -P solo -n worker1@%h ``` worker2 ``` celery -A tasks worker --loglevel=info --concurrency=1 -P solo -n worker2@%h ``` observe ``` Refusing to deserialize disabled content of type msgpack (application/x-msgpack) ``` Repeated in logs of both workers every 2 seconds Now set ``` event_serializer = 'json' ``` and observe no more error

Expected Behavior

No error message regarding msgpack serialization

Actual Behavior

Refusing to deserialize disabled content of type msgpack (application/x-msgpack)

In logs every 2 seconds on both workers

I am not sure of the effects on the execution of the program, but I think I have narrowed it down to event serialization. Docs at https://docs.celeryq.dev/en/latest/userguide/configuration.html#std-setting-event_serializer do not explicitly state if msgpack is supported for events or not

open-collective-bot[bot] commented 1 year ago

Hey @thomas-thorburn-connect :wave:, Thank you for opening an issue. We will get back to you as soon as we can. Also, check out our Open Collective and consider backing us - every little helps!

We also offer priority support for our sponsors. If you require immediate assistance please consider sponsoring us.

MemoryLeak55 commented 1 year ago

Any news on this?

auvipy commented 1 year ago

this issue seems to be related https://github.com/celery/kombu/issues/1065

sondrelg commented 1 year ago

Could be that you need to set CELERY_ACCEPT_CONTENT = ['msgpack', ...]. That's required when using msgpack as a serializer.

MemoryLeak55 commented 1 year ago

Could be that you need to set CELERY_ACCEPT_CONTENT = ['msgpack', ...]. That's required when using msgpack as a serializer.

Wouldn't my test case settings be sufficient?

class Config:
    event_serializer = 'msgpack'
    task_serializer = "msgpack"
    result_serializer = "msgpack"
    accept_content = ["msgpack"]
    result_accept_content = ["msgpack"]

Or is the all caps config item important?

sondrelg commented 1 year ago

Sorry didn't see the example, just thought I'd drop in in case it was helpful 🙇 Perhaps check that the settings load correctly? I think the serializers and other settings are displayed when you start celery