Airflow 3.0 deprecations

astral-sh / ruff

An extremely fast Python linter and code formatter, written in Rust.

https://docs.astral.sh/ruff

MIT License

33.37k stars 1.11k forks source link

Airflow 3.0 deprecations #14626

Open dhruvmanila opened 1 week ago

dhruvmanila commented 1 week ago

Tracking issue for the AIR3xx rules that implements Airflow 3.0 deprecations.

For details, refer to the Airflow tracking issue: https://github.com/apache/airflow/issues/44556

Ref: https://github.com/apache/airflow/issues/41641

Lee-W commented 6 days ago

There's already a short proposal https://github.com/astral-sh/ruff/pull/14616#issuecomment-2503207647. Putting it here might make it easier to find 🙂

I'm thinking of separating "removal" and "rename" at least. It would be much easier for the users to know whether there's anything they could do to fix it.

dhruvmanila commented 6 days ago

Additionally, the NumPy 2.0 migration rule is a good reference whose implementation is at: https://github.com/astral-sh/ruff/blob/6f1cf5b686e67fb74383641fcbbe98e33481a271/crates/ruff_linter/src/rules/numpy/rules/numpy_2_0_deprecation.rs

Lee-W commented 4 days ago

Hi, we have grouped our deprecations into several categories and would appreciate it if the team could take a look and let us know what you think about it 🙂

already merged
- AIR301: Avoid implicit DAG schedule https://github.com/astral-sh/ruff/pull/14581
newly proposed error number
- AIR302: removal or no direct mapping
  - following https://github.com/astral-sh/ruff/pull/14582
- AIR303: rename
  - It would be much easier for the users to know whether they could do anything to fix it.
- AIR304: moved to airflow providers
  - for these, they're similar to "rename" but require the user to install an additional package (e.g., apache-airflow-providers-fab)

Here's the categorised list we have now https://github.com/apache/airflow/issues/41641#issuecomment-2508140120 (there's another group we temporarily mark as AIR305, but we'll need some time to figure out what we can do for those. Will likely to be merged into 302 or 303)

MichaReiser commented 4 days ago

I like what's proposed here. Thanks for going through all the deprecations and grouping them by possible rules. We can also do some fine-tuning as we go.

Lee-W commented 1 day ago

Later discuss with @uranusjr, we think it would be better for us to merge rename and removal into 302. So the current idea would be

already merged
- AIR301: Avoid implicit DAG schedule (https://github.com/astral-sh/ruff/pull/14581)
- AIR302: removal (https://github.com/astral-sh/ruff/pull/14582)
new rule number
- AIR303: moved to airflow providers
  - for these, they're similar to "rename" but require the user to install an additional package (e.g., apache-airflow-providers-fab)

We now use https://github.com/apache/airflow/issues/44556 to track the detail of this task instead of the original one as some of the migration might not be able to be done in ruff

MichaReiser commented 7 hours ago

Sorry for pinging you here, but could someone more familiar with Airflow volunteer looking at the changes in https://github.com/astral-sh/ruff/pull/14631? We just want to make sure that the change is correct.

Lee-W commented 6 hours ago

Sorry for pinging you here, but could someone more familiar with Airflow volunteer looking at the changes in #14631? We just want to make sure that the change is correct.

Looks like a valid one to me

MichaReiser commented 6 hours ago

Thanks for taking a look!