search

apache / airflow-on-k8s-operator

Airflow on Kubernetes Operator
https://airflow.apache.org/
Apache License 2.0
89 stars 26 forks source link

Why architecture is different from configmap? #11

Open Rep1AI opened 4 years ago

Rep1AI commented 4 years ago

As you can see https://github.com/apache/airflow-on-k8s-operator/blob/master/templates/airflow-configmap.yaml#L37 executor = KubernetesExecutor

But in it shows CeleryExecutor: https://github.com/apache/airflow-on-k8s-operator/blob/master/docs/airflow-cluster.png

Lot of confusions..

Basic questions:

If yes, Where is docs on installing multi node airflow with KubernetesExecutor?

barney-s commented 4 years ago

config map is used only if Executor type is K8s in the CRD https://github.com/apache/airflow-on-k8s-operator/blob/dd430eaa7c3add972f410e058a2fe0a4a58eeb64/controllers/airflowcluster_controller.go#L726

It does support it but needs some TLC and may be broken: https://github.com/apache/airflow-on-k8s-operator/blob/master/hack/sample/mysql-k8s/cluster.yaml

Rep1AI commented 4 years ago

needs some TLC and may be broken

Sorry didn't understand, What is TLC? and the operator currently is not stable to use?

GrigorievNick commented 4 years ago

I have the same question here. Look like, the operator does not use celery any more. But use k8s Executor. Can some one confirm this?