apache / airflow

Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
https://airflow.apache.org/
Apache License 2.0
35.21k stars 13.76k forks source link

"Queue up new tasks" equivalent in CLI / REST API #38342

Open qcha41 opened 3 months ago

qcha41 commented 3 months ago

Description

Suggestion to add a new command in the CLI / REST API to reproduce the "Queue up new tasks" action that can be trigger within the web UI, with start_date and end_date parameters to allow running this action accross all dag runs lying a given period.

For the CLI, it could be airflow dags queue_up_new_tasks or airflow dags reload for instance.

Use case/motivation

Right now, when we add new tasks to an existing dag containing existing dag runs, it is not straightforward to run these newly added tasks in the existing/previous dag runs.

Two workarounds :

Or maybe there are already solutions par that ?

Related issues

https://github.com/apache/airflow/discussions/30435#discussioncomment-8856597

Are you willing to submit a PR?

Code of Conduct

bbovenzi commented 3 months ago

+1 We should move the UI to use the rest api endpoint for this too