stackabletech / airflow-operator

Stackable Operator for Apache Airflow
Other
21 stars 2 forks source link

requirement of celery configs when using kubernetes executors #424

Closed maxgruber19 closed 4 months ago

maxgruber19 commented 5 months ago

Affected Stackable version

24.3.0

Affected Apache Airflow version

2.7.2

Current and expected behavior

airflow pods fail to start with CreateContainerConfigError because of missing celery configs in the credentials secret even though celery is not used anymore since it's replaced by kubernetes executors.

Error: couldn't find key connections.celeryBrokerUrl in Secret usecase1/airflow-credentials Error: couldn't find key connections.celeryResultBackend in Secret usecase1/airflow-credentials

i expect airflow to start up without celery configs because i dont want to have an unneccessary redis deployment running in the cluster. seems like faking the configs works, but it's hacky

Possible solution

No response

Additional context

No response

Environment

Client Version: v1.28.7 Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3 Server Version: v1.27.10

Would you like to work on fixing this bug?

None