Closed Timelessprod closed 3 weeks ago
tirkarthi
commented
3 weeks ago main branch is now under development for Airflow 3 where migration files were pruned. You should probably try using v2-10-test if fixes for 2.10 are needed and use 2.10.0 tag.
Timelessprod
commented
3 weeks ago @tirkarthi Why have those version files been removed from the 2.10.0 release ? Currently Alembic cannot find them so there's something wrong with the release I think.
Is this "v2-10-test" version among the pre-releases available on PIP ? My libraries are managed by PIP and I'd prefer no to manually clone the repo if possible.
Thank you.
potiuk
commented
3 weeks ago You have not mentioned it but my guess is that you are dynamically installing packages and after this dynamic installation - ikely your image actually runs lower version of Airlfow.
The revision is in the alembic table so the migration worked - but the airflow command you run uses a downgraded version of airflow after your dynamic pip install command.
Please take a look on what version of the image you are using and whether your customization / extending the image has some problems. For example when you are installing dynamically new packages - the pip install command might automatically downgrade airflow. It solely depends on packages you install and resolution of pip which might decide that downgrading airflow fulfills other dependencies better.
That's why in the offiical installation description of upgrade scenarios, you will find that you always have to specify apache-airflow=${AIRFLOW_VERSION} https://airflow.apache.org/docs/apache-airflow/stable/installation/installing-from-pypi.html#installation-and-upgrade-scenarios - so that pip won't downgrade airflow.
This is one of the reasons why we always discouraged installing packages dynamically when your image starts - because for example airlfow version might change - but also other packages might and recommend users to build their own image - this way they can be sure what version of packages they have there. With dynamic package
Apache Airflow version
2.10.0
If "Other Airflow 2 version" selected, which one?
No response
What happened?
When upgrading Airflow from version 2.8.3 to 2.10.0, Airflow fails to restart. Indeed the Alembic migration fails with the error:
The missing version file is located here : https://github.com/apache/airflow/blob/e001b88f5875cfd7e295891a0bbdbc75a3dccbfb/airflow/migrations/versions/0148_2_10_0_dataset_alias_dataset_event.py
It's no longer on the master branch but it is on the last commit with tag
2.10.0In a attempt to fix the problem I tried to revert the version back to 2.8.3 but Alembic is still looking for the
ec3471c1e067version file when restarting the instance, and thus Airflow is unable to start anymore.What you think should happen instead?
The migration should have go through without error and Airflow start.
How to reproduce
Operating System
Debian 12.6
Versions of Apache Airflow Providers
From the official constraints file at https://raw.githubusercontent.com/apache/airflow/constraints-2.10.0/constraints-3.11.txt
Deployment
Other 3rd-party Helm chart
Deployment details
Custom Helm chart deployment.
Anything else?
Currently my instance of Airflow is unable to start, even after trying to revert to the last used version (2.8.3).
Are you willing to submit PR?
Code of Conduct