Open 0x26res opened 2 days ago
Other Airflow 2 version (please specify below)
2.8.1 (MWAA)
My airflow deferred (AWS) BatchOperator occasionally fail. When this happens I can see this:
{{local_task_job_runner.py:302}} WARNING - State of this instance has been externally set to deferred. Terminating instance.
I actually think it's an oversight in the code for local_task_job_runner. This line should check for state being RUNNING or DEFERRED.
No response
This is hard to reproduce. The issue is very transient.
MWAA
apache-airflow-providers-amazon==8.16.0
Official Apache Airflow Helm Chart
I would say it happens one out of 100 run.
Apache Airflow version
Other Airflow 2 version (please specify below)
If "Other Airflow 2 version" selected, which one?
2.8.1 (MWAA)
What happened?
My airflow deferred (AWS) BatchOperator occasionally fail. When this happens I can see this:
I actually think it's an oversight in the code for local_task_job_runner. This line should check for state being RUNNING or DEFERRED.
What you think should happen instead?
No response
How to reproduce
This is hard to reproduce. The issue is very transient.
Operating System
MWAA
Versions of Apache Airflow Providers
apache-airflow-providers-amazon==8.16.0
Deployment
Official Apache Airflow Helm Chart
Deployment details
MWAA
Anything else?
I would say it happens one out of 100 run.
Are you willing to submit PR?
Code of Conduct