Closed dacort closed 1 year ago
Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.
Thank you for creating the issue @dacort , would you like to create a PR for the on_kill
method implementation?
@phanikumv Yep, happy to! Just getting reacquainted with my airflow dev environment. 😁
Assigned it to you
When marking an EMR Serverless job as failed, the job continues to run.
This is expected. Marking a task as failed in the UI just change the status of the task in the DB. It does not invoke on_kill function.
Converting to feature (add on_kill() function) as this is not a bug.
It does not invoke on_kill function.
I was wondering if that was the case. When does it get invoked?
This is expected. Marking a task as failed in the UI just change the status of the task in the DB. It does not invoke on_kill function.
I was wondering if that was the case. When does it get invoked?
I think @eladkal you mistook it with something else (some callbacks or maybe race conditions ?). When task is running and you clear it from the UI, it will be marked as FAILED in the DB and when the task runs a heartbeat, as far as I know it checks if the state is set to "FAILED" and generally it will run the "on_kill" method when exiting. So I think indeed it is a bug.
yes we need to add on_kill method on the operator class to kill stale processes on target system. For example we did the same in Trino Operator when the task is cleared from UI and the query wasnt getting killed in Trino DB
Apache Airflow version
2.6.0
What happened
When marking an EMR Serverless job as failed, the job continues to run.
What you think should happen instead
The job should be cancelled. Looking at the EMR Serverless Operator, I don't see an
on_kill
method, so assuming we just need to add that.I'm not sure how to handle the
EmrServerlessCreateApplicationOperator
operator, though - if the workflow has a correspondingEmrServerlessDeleteApplicationOperator
, we'd probably want to delete the application if the job is cancelled.How to reproduce
Operating System
n/a
Versions of Apache Airflow Providers
apache-airflow-providers-amazon==8.0.0
Deployment
Amazon (AWS) MWAA
Deployment details
No response
Anything else
No response
Are you willing to submit PR?
Code of Conduct