Include parse-url support for rabbitmq (amqp://) scheme

[timotei-za]

Problem: Singleton base are not supported if rabbitmq broker are used to assimilate the tasks. Error: Redis URL must specify one of the following schemes (redis://, rediss://, unix://)

If there is another way to use singleton with the rabbitmq broker, I would be very grateful to know. Thanks!

recap: the problem is that when I use rabbitmq as a broker, I can't use Singleton property inside of decorator as a base, because parse_url method doesn't support the rabbitmq system

env.config file:

CELERY_BROKER_HOST=redis://127.0.0.1:6379

tasks.py file:

@app.task(base=Singleton, name="create_or_update_user")
def create_or_update(user_id: str):
    data = async_to_sync(update_or_create_user)(user_id=user_id)

    return data

create_user.py file:

data = create_or_update.apply_async(
       args=[user_id],
       countdown=CELERY_TASKS_COUNTDOWN, 
)

[steinitzu]

Singleton requires redis for the locking backend, but redis does not need to be your broker.

Set CELERY_SINGLETON_BACKEND_URL to a valid redis url. You can then use rabbitmq as a broker.

[timotei-za]

@steinitzu same. This doesn't work!

[timotei-za]

@steinitzu Do you have another proposal? 😃

[jdavid]

Had the same error. This worked for me (Django example, using old Celery 4.3.1):

app.config_from_object('django.conf:settings', namespace='CELERY')
app.conf.singleton_backend_url = settings.CELERY_SINGLETON_BACKEND_URL