Suggestion to add a new command in the CLI / REST API to reproduce the "Queue up new tasks" action that can be trigger within the web UI, with start_date and end_date parameters to allow running this action accross all dag runs lying a given period.
For the CLI, it could be airflow dags queue_up_new_tasks or airflow dags reload for instance.
Use case/motivation
Right now, when we add new tasks to an existing dag containing existing dag runs, it is not straightforward to run these newly added tasks in the existing/previous dag runs.
Two workarounds :
airflow dags backfill but not satisfying as it transforms "scheduled" existing dags run into "backfill" dag runs, which prevent us from interacting with them through the web UI (like clearing tasks, etc.)
airflow tasks render followed by airflow tasks clear but it is not convenient as the render command is only working for one specific execution_date (or run_id) at a time.
Description
Suggestion to add a new command in the CLI / REST API to reproduce the "Queue up new tasks" action that can be trigger within the web UI, with
start_date
andend_date
parameters to allow running this action accross all dag runs lying a given period.For the CLI, it could be
airflow dags queue_up_new_tasks
orairflow dags reload
for instance.Use case/motivation
Right now, when we add new tasks to an existing dag containing existing dag runs, it is not straightforward to run these newly added tasks in the existing/previous dag runs.
Two workarounds :
airflow dags backfill
but not satisfying as it transforms "scheduled" existing dags run into "backfill" dag runs, which prevent us from interacting with them through the web UI (like clearing tasks, etc.)airflow tasks render
followed byairflow tasks clear
but it is not convenient as therender
command is only working for one specific execution_date (or run_id) at a time.Or maybe there are already solutions par that ?
Related issues
https://github.com/apache/airflow/discussions/30435#discussioncomment-8856597
Are you willing to submit a PR?
Code of Conduct