Rotating broker password support

[arbroen]

Checklist

• [x] I have checked the issues list for similar or identical feature requests.
• [x] I have checked the pull requests list for existing proposed implementations of this feature.
• [x] I have checked the commit log to find out if the if the same feature was already implemented in the master branch.
• [x] I have included all related issues and possible duplicate issues in this issue (If there are none, check this box anyway).

Related Issues and Possible Duplicates

Related Issues

• None

Possible Duplicates

• BROKER_LOGIN_METHOD Seems to offer some sort of alternate login method, however the documentation tells me it is only meant for AMQP styled brokers. Pull request.

Brief Summary

Modern key vaults like Azure Key Vault support key/secret rotation. In my personal case the TTL of a secret value is about ~2 hours, with a little random component. This means creating new connections to a BROKER_URL after the key has been rotated will fail.

The current approach declares the BROKER_URL as a static variable in the settings. Key rotation would need to have at least the password part of the design to be dynamic.

This feature would increase security options when using Celery. Stimulate the user to tread carefully.

Design

Architectural Considerations

Proposed Behavior

The ability to include a BROKER_PASSWORD_TTL. As soon as the time to live expires a rotation callback function is used to get the new password value.

Example user implementation:

# Celery.py
import datetime

delta = datetime.timedelta(hours=2)

Celery(
    BROKER_URL='redis://:secrets@example.com:1234', 
    BROKER_PASSWORD_TTL=delta, 
    BROKER_PASSWORD_ROTATOR='module.submodule:ttl_function')

And the rotator:

def ttl_function(**kwargs) -> Tuple[str, datetime]:
    """Responsible for retrieving the new key. Returns the new key and new expiration date."""
    new_time = datetime.datetime.utcnow() + interval
    return get_azure_secret("REDIS_URL"), new_time

Proposed UI/UX

BROKER_PASSWORD_TTL - A timedelta, datetime or crontab when the password expires. BROKER_PASSWORD_ROTATOR - function that handles password rotation (as such the user can implement any provider/custom wishes themselves initially).

Diagrams

N/A

Alternatives

• A more rudimentary callback function for when the BROKER returns an authentication error. This callback would be supplied with the error and other values and the user can determine based on these values whether he wants to retry with a new password for example.
• A periodic task could be implemented by the user that updates the configuration key at run-time. However, this is an entirely custom implementation whereas and I quote: "Celery is written with security in mind". link.

Thank you

For maintaining this awesome library. PS. If someone points me in the right direction I would be very much willing to create a PR for this.

[thedrow]

We're currently in the architecture phase of Celery 5.0.0.

I'd rather not introduce new features to 4.x which may complicate it further. We already have connection problems with brokers as it is.

If you'd like to get involved, ping me on twitter or drop me an email.

[VigneswaranB97]

Im trying to use rabbitmq as a broker for celery and using Hashicorp vault for root credential rotation. How do I use Hashicorp vault's rabbitmq secret engine with celery? Since this issue is related, Is there any updates on this for version 5.2.7.

[SorianoMarmol]

Any news on this? Thank you very much.

[gabloe]

Any update to this?