search

apache / airflow

Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
https://airflow.apache.org/
Apache License 2.0
35.21k stars 13.76k forks source link

Ensure try_number incremented for empty operator #40426

Closed dstandish closed 2 days ago

dstandish commented 3 days ago

Missed this when implementing #39336. Need to increment try number for empty operator too.

vincbeck commented 2 days ago

Interesting, I did not know we where making an exception for EmptyOperator. What is the reason behind? Just optimization purposes? I am just wondering if it is worth it and we should not just treat it as any regular operator

dstandish commented 2 days ago

Interesting, I did not know we where making an exception for EmptyOperator. What is the reason behind? Just optimization purposes? I am just wondering if it is worth it and we should not just treat it as any regular operator

Yeah. I don’t have a strong opinion. I would not be opposed to removing the logic. +0 you could say. I think it’s there because why send something to the worker to do nothing. But then we have all that logic cus if there’s callbacks etc

I don’t know why we need this operator in the first place. I’m not sure what the use is.

jedcunningham commented 2 days ago

I love me some EmptyOp, but that's for my persona (Airflow dev). I'd bet some 🍻 that the vast majority of EmptyOps in the wild could go away.