search

mumoshu / kube-airflow

A docker image and kubernetes config files to run Airflow on Kubernetes
Apache License 2.0
655 stars 205 forks source link

Embedded dags - handling deployments #31

Open darrenhaken opened 5 years ago

darrenhaken commented 5 years ago

I have just got Airflow working on K8 through the help of this great project, thanks!

I had a question about the embedded DAG's approach. In the event of a deployment or a pod being rescheduled by K8, how would a running DAG survive the process?

Or is the answer quite simply it wouldn't and it would need to retry?

Any other wise words on having a reliable Airflow on K8 are welcome

liangjun-jiang commented 5 years ago

I can assume this question has been answer at some where. but still looking for an answer.