airflow-helm / charts

The User-Community Airflow Helm Chart is the standard way to deploy Apache Airflow on Kubernetes with Helm. Originally created in 2017, it has since helped thousands of companies create production-ready deployments of Airflow on Kubernetes.
https://github.com/airflow-helm/charts/tree/main/charts/airflow
Apache License 2.0
630 stars 474 forks source link

Support for other drivers for MySQL database #798

Open turuncuofke opened 8 months ago

turuncuofke commented 8 months ago

Checks

Motivation

I am currently using airflow version "2.7.0-python3.11" with the chart version "8.8.0" and I am using an external MySQL database. I am getting _"WARNING: MYSQL_OPTRECONNECT is deprecated and will be removed in a future version." logs from my airflow cluster.

After some research, I realized that these warning logs are coming because of the database driver I use. When I wanted to change the driver I use for database connection to "mysql-connector-python" from "mysqlclient" I had some problems with changing the database connection string since it is given as DATABASE_SQLALCHEMY_CMD: {{ `echo -n "mysql+mysqldb://$(eval $DATABASE_USER_CMD):$(eval $DATABASE_PASSWORD_CMD)@${DATABASE_HOST}:${DATABASE_PORT}/${DATABASE_DB}${DATABASE_PROPERTIES}"` | b64enc | quote }} with hardcoded database driver in the "secret-configs-env-yaml" file in the chart.

As a workaround, I overrode this "DATABASE_SQLALCHEMY_CMD" env variable using airflow.extraEnv in the values.yaml as:

Screenshot 2023-10-23 at 11 15 21

Since it is available in Airflow 2.7.0 https://airflow.apache.org/docs/apache-airflow/2.7.0/howto/set-up-database.html#setting-up-a-mysql-database

Could this driver parameter in database connection string also be a variable which can be set in the values.yaml?

Implementation

No response

Are you willing & able to help?