Investigate better DAG version detection?

apache / airflow

Apache Airflow - A platform to programmatically author, schedule, and monitor workflows

https://airflow.apache.org/

Apache License 2.0

36.71k stars 14.21k forks source link

Investigate better DAG version detection? #42337

Open jedcunningham opened 4 weeks ago

jedcunningham commented 4 weeks ago

Body

Do this part of the AIP:

We will explore having a more comprehensive version detection method (e.g. if the code inside of a PythonOperators python_callable changes, or in functions that that callable calls)

Simply detecting when the python_callable that is passed changes (somehow), might be enough and a good line.

Committer

[x] I acknowledge that I am a maintainer/committer of the Apache Airflow project.

kunaljubce commented 3 weeks ago

@jedcunningham If I understand the ask correctly, as of today, the DAG UI still shows the older version of the python_callable code and we need a way to identify/log when the code change happens and show the updated code on the UI. Am I correct?