When Airflow is configured to use GCP Secret Manager as the backend for variable storage, Variable.get() retrieves the latest version of the secret from Secret Manager. However, Variable.set() currently creates a variable in the Airflow database instead of creating a new version in Secret Manager.
This inconsistency can lead to confusion and potential security risks.
I propose enhancing Variable.set() method to add the possibility to create a new version of the secret in Secret Manager when the backend is connected, maintaining consistency with Variable.get() behavior.
Use case/motivation
Consistency: Users expect Variable.set() to interact with Secret Manager when it's configured as the backend, just as Variable.get() does.
Version control: Creating new versions in Secret Manager allows for better tracking and management of variable changes over time.
Security: Storing variables directly in Secret Manager, rather than the Airflow database, maintains the security benefits of using a dedicated secret management service.
Description
When Airflow is configured to use GCP Secret Manager as the backend for variable storage, Variable.get() retrieves the latest version of the secret from Secret Manager. However, Variable.set() currently creates a variable in the Airflow database instead of creating a new version in Secret Manager.
This inconsistency can lead to confusion and potential security risks.
I propose enhancing Variable.set() method to add the possibility to create a new version of the secret in Secret Manager when the backend is connected, maintaining consistency with Variable.get() behavior.
Use case/motivation
Related issues
None that I'm aware of.
Are you willing to submit a PR?
Code of Conduct