The User-Community Airflow Helm Chart is the standard way to deploy Apache Airflow on Kubernetes with Helm. Originally created in 2017, it has since helped thousands of companies create production-ready deployments of Airflow on Kubernetes.
Client Version: v1.30.2
Server Version: v1.30.6-eks-7f9249a
### Helm Version
```shell
version.BuildInfo{Version:"v3.16.2", GitCommit:"13654a52f7c70a143b1dd51416d633e1071faffb", GitTreeState:"clean", GoVersion:"go1.22.7"}
### Description
I was attempting to start Airflow using Helm and using an external AWS RDS PostgreSQL instance as the database. Pods such as the worker and triggerer hang with the logs:
Created container wait-for-airflow-migrations
Started container wait-for-airflow-migrations
I have proved that the RDS instance is reachable from within the cluster, and I have manually created the database user and tables in an effort to solve this issue.
### Relevant Logs
```shell
Output of `kubectl get pods -n airflow`:
airflow-redis-0 1/1 Running 0 12m
airflow-run-airflow-migrations-7c4qd 1/1 Running 4 (112s ago) 12m
airflow-scheduler-d667c946d-mljlx 1/2 Running 7 (3m45s ago) 12m
airflow-statsd-b45f54fb4-f5vgg 1/1 Running 0 12m
airflow-triggerer-0 0/2 Init:CrashLoopBackOff 3 (15s ago) 12m
airflow-webserver-75d89cb74d-68bz7 0/1 CrashLoopBackOff 7 (13s ago) 12m
airflow-worker-0 0/2 Init:CrashLoopBackOff 3 (14s ago) 12m
Output of kubectl describe pod airflow-triggerer-0 -n airflow:
Normal Pulled 6s (x5 over 12m) kubelet Container image "apache/airflow:2.9.3" already present on machine
Normal Created 6s (x5 over 12m) kubelet Created container wait-for-airflow-migrations
Normal Started 6s (x5 over 12m) kubelet Started container wait-for-airflow-migrations
Custom Helm Values
postgresql:
## to use the external db, the embedded one must be disabled
enabled: false
## for full list of PgBouncer configs, see values.yaml
pgbouncer:
enabled: true
externalDatabase:
type: postgres
host: <hostname>
port: 5432
## the database/scheme to use within the external database
database: airflow_db
## (username - option 1) a plain-text helm value
user: airflow_user
password: <password here>
triggerer:
persistence:
storageClassName: dev-ebs-sc-hot-storage
workers:
persistence:
storageClassName: dev-ebs-sc-hot-storage
redis:
persistence:
storageClassName: dev-ebs-sc-hot-storage
dags:
persistence:
storageClassName: dev-ebs-sc-hot-storage
logs:
persistence:
storageClassName: dev-ebs-sc-hot-storage
Checks
User-Community Airflow Helm Chart
.Chart Version
1.15.0
Kubernetes Version
Created container wait-for-airflow-migrations Started container wait-for-airflow-migrations
Output of
kubectl describe pod airflow-triggerer-0 -n airflow
:Custom Helm Values