apache / airflow

Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
https://airflow.apache.org/
Apache License 2.0
36.49k stars 14.13k forks source link

Difficulty with Clearing and Restarting Tasks in Airflow UI for bakcfill DAGs #41468

Open ImanLakjiri opened 1 month ago

ImanLakjiri commented 1 month ago

Apache Airflow version

2.10.0rc1

If "Other Airflow 2 version" selected, which one?

No response

What happened?

I executed the following command to backfill Airflow DAGs:

airflow dags backfill -s "$current_date" -e "$next_day" "$DAG_ID" --reset-dagruns However, I encountered several issues:

I cannot clear tasks from the Airflow UI. If the backfill process is interrupted or killed, I cannot restart or resume the DAG from the UI.

What you think should happen instead?

No response

How to reproduce

Run the backfill command:

airflow dags backfill -s "$current_date" -e "$next_day" "$DAG_ID" --reset-dagruns Check the Airflow UI:

Attempt to clear tasks and verify if the option is functional. Try to restart or resume the DAG if the process is killed.

Operating System

cat /etc/os-release

Versions of Apache Airflow Providers

Version: v2.8.0

Deployment

Official Apache Airflow Helm Chart

Deployment details

No response

Anything else?

No response

Are you willing to submit PR?

Code of Conduct

boring-cyborg[bot] commented 1 month ago

Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.

vatsrahul1001 commented 1 month ago

Verify that this issue persists, but note that it wasn't introduced with 2.10.rc1; it also existed in 2.9.3.

GabrielEisenbergOlympus commented 1 month ago

I have also experienced this issue. After tasks were backfilled and then cleared over the same time period, a clear does not initiate queuing of the tasks. Note that this existed in both 2.8.1 and now in 2.9.2 in MWAA.

image