search

apache / airflow

Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
https://airflow.apache.org/
Apache License 2.0
36.34k stars 14.09k forks source link

Unable to Stop / Kill the currently running Airflow DAG programmatically. #10704

Closed jsaradhy closed 1 year ago

jsaradhy commented 4 years ago

I have an usecase, we are implementing the automation with Gitlab, AWS S3 & Codepipeline, Code & Airflow. In a nutshell when there is a new version of the dag is available in the DAG BAG, i need to pro-grammatically stop the currently running Airflow DAG.

Gurus, could you please shed some light on this : Programmatically how can i stop/ kill the currently running DAG in Airflow ?

boring-cyborg[bot] commented 4 years ago

Thanks for opening your first issue here! Be sure to follow the issue template!

msumit commented 4 years ago

It feels like a use case of DAG versioning, with a flag to let scheduler/worker to kill the running tasks if they are not at the latest version.

@kaxil should be able to shed more light here.

pchtsp commented 3 years ago

Hello, I just found this issue and I was wondering if this is the same need as mine. I would like to be able to kill a running dagrun in a worker by sending some signal. Ideally with a dedicated endpoint in the rest api but it can also be in some other way indirect way as changing some flag in the airflow database.

eladkal commented 2 years ago

It feels like a use case of DAG versioning, with a flag to let scheduler/worker to kill the running tasks if they are not at the latest version.

@kaxil should be able to shed more light here.

Regardless of versioning user might want to kill a process for diffrent reasons.

We do have endpoint to change DagRun state I think this won't invoke the on_kill() method of the tasks that are currently running.