search

conda-forge / airflow-feedstock

A conda-smithy repository for airflow.
BSD 3-Clause "New" or "Revised" License
6 stars 17 forks source link

papermill #68

Closed afonit closed 3 years ago

afonit commented 3 years ago

I am looking to upgrade to the 2.x of airflow where I used it with papermill.

I am no longer seeing papermill as an option, I do see it mentioned here: https://github.com/conda-forge/airflow-feedstock/pull/66

but if I do:

conda install -c conda-forge airflow-with-papermill

it never resolves. (which makes sense as I don't see it listed on here https://github.com/conda-forge/airflow-feedstock)

Is there a way to get papermill added to the conda packaging of airflow 2.x?

xylar commented 3 years ago

Hi @afonit, Apache Airflow has changed the way that you get packages for specific providers. You now install a separate package (which has its own feedstock). In this case, you want apache-airflow-providers-papermill. See: https://airflow.apache.org/docs/apache-airflow-providers-papermill/stable/index.html https://github.com/conda-forge/apache-airflow-providers-papermill-feedstock

xylar commented 3 years ago

One more note. It will likely be safer to create a new conda environment with conda create -n <name> -c conda-forge apache-airflow-providers-papermill than to try to upgrade an existing environment. Better still, we recommend adding conda-forge as a default channel because mixing packages from different channels can have unexpected consequences:

conda config --add channels conda-forge
conda config --set channel_priority strict
conda create -n <name> apache-airflow-providers-papermill