celery / celery

Distributed Task Queue (development branch)
https://docs.celeryq.dev
Other
24.79k stars 4.67k forks source link

Same task is copied multiple times and exceeding the `max_retries` parameter value #6931

Open langdangren opened 3 years ago

langdangren commented 3 years ago

Checklist

Mandatory Debugging Information

Optional Debugging Information

Related Issues and Possible Duplicates

Related Issues

Possible Duplicates

Environment & Settings

Celery version: 5.1.2 (sun-harmonics)

celery report Output:

```shell software -> celery:5.1.2 (sun-harmonics) kombu:5.1.0 py:3.9.6 billiard:3.6.4.0 redis:3.5.3 platform -> system:Darwin arch:64bit kernel version:19.6.0 imp:CPython loader -> celery.loaders.app.AppLoader settings -> transport:redis results:redis://localhost:6379/0 broker_url: 'redis://localhost:6379/0' result_backend: 'redis://localhost:6379/0' deprecated_settings: None ```

Steps to Reproduce

Python Packages

pip freeze Output:

``` amqp==5.0.6 billiard==3.6.4.0 celery==5.1.2 certifi==2021.5.30 click==7.1.2 click-didyoumean==0.0.3 click-plugins==1.1.1 click-repl==0.2.0 kombu==5.1.0 prompt-toolkit==3.0.20 pytz==2021.1 redis==3.5.3 six==1.16.0 vine==5.0.0 wcwidth==0.2.5 ```

Minimally Reproducible Test Case

```python from celery import Celery app = Celery("celery_app", broker="redis://localhost:6379/0", backend="redis://localhost:6379/0") @app.task(bind=True, max_retries=10) def work(self): try: raise self.retry(countdown=2) except Exception: raise self.retry(countdown=2) if __name__ == '__main__': res = work.delay() print(res) ```

Expected Behavior

The task terminates when the max_retries=10 of retries is reached,And then throw an celery.exceptions.MaxRetriesExceededError

Actual Behavior

Multiple repeated tasks were received, and the number of tasks greatly exceeded the max_retries value. Procedure This causes the task to be executed multiple times. In addition, after testing, I found that the number of replication tasks increases with the increment of countdown.

image image image image

open-collective-bot[bot] commented 3 years ago

Hey @TryReboot :wave:, Thank you for opening an issue. We will get back to you as soon as we can. Also, check out our Open Collective and consider backing us - every little helps!

We also offer priority support for our sponsors. If you require immediate assistance please consider sponsoring us.