search

apache / airflow

Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
https://airflow.apache.org/
Apache License 2.0
37.26k stars 14.33k forks source link

Do not add "extra" if provider is already required dependency #44195

Open potiuk opened 2 days ago

potiuk commented 2 days ago

When we see a providers is a dependency in "cross-provider-dependecies" - we add it automatically - when we build the package - as optional extra. However a number of providers also have the same provider as required dependency, and in this case adding an extra makes little sense.

Example https://airflow.apache.org/docs/apache-airflow-providers-jdbc/stable/index.html#cross-provider-package-dependencies - JDBC provider has "common.sql" extra but it also have "common.sql" as required dependency.

If that is the case, we should skip generating extra when the provider is prepared. This happens in https://github.com/apache/airflow/blob/81a910db9af72db0c7d12c33bb186cb8b117322e/dev/breeze/src/airflow_breeze/utils/packages.py#L471

amoghrajesh commented 2 days ago

Assigning to self. Will take a stab at it soon.