conda-forge / airflow-feedstock

A conda-smithy repository for airflow.
BSD 3-Clause "New" or "Revised" License
6 stars 17 forks source link

Airflow extras don't work with python > 2.7 #18

Closed halldc closed 6 years ago

halldc commented 6 years ago

Installing vanilla airflow with py36 works just fine: $ conda create -n test python=3.6.6 airflow=1.10

But installing airflow with extras fails:

$ conda create -n test python=3.6.6 airflow-with-celery=1.10
Solving environment: failed

UnsatisfiableError: The following specifications were found to be in conflict:
  - airflow-with-celery=1.10 -> airflow==1.10.0=py27_0 -> funcsigs==1.0.0 -> python=2.7 -> openssl=1.0.1
  - airflow-with-celery=1.10 -> airflow==1.10.0=py27_0 -> funcsigs==1.0.0 -> python=2.7 -> readline=6.2
  - airflow-with-celery=1.10 -> airflow==1.10.0=py27_0 -> funcsigs==1.0.0 -> python=2.7 -> tk=8.5
  - python=3.6.6

It looks like airflow-with-celery=1.10 has been pinned to the py27 version of airflow=1.10, but this should be independent of python version.

@ocefpaf @marcelotrevisani @sodre Have you experienced this issue before? Should I be making these split packages noarch? They don't actually install any code themselves - just list the dependencies they need.


Details about conda and system ( conda info ):

``` $ conda info active environment : None shell level : 0 user config file : /Users/david.hall/.condarc populated config files : /Users/david.hall/.condarc conda version : 4.5.11 conda-build version : 3.12.1 python version : 3.6.5.final.0 base environment : /Users/david.hall/miniconda3 (writable) channel URLs : https://conda.anaconda.org/conda-forge/osx-64 https://conda.anaconda.org/conda-forge/noarch https://repo.anaconda.com/pkgs/main/osx-64 https://repo.anaconda.com/pkgs/main/noarch https://repo.anaconda.com/pkgs/free/osx-64 https://repo.anaconda.com/pkgs/free/noarch https://repo.anaconda.com/pkgs/r/osx-64 https://repo.anaconda.com/pkgs/r/noarch https://repo.anaconda.com/pkgs/pro/osx-64 https://repo.anaconda.com/pkgs/pro/noarch package cache : /Users/david.hall/miniconda3/pkgs /Users/david.hall/.conda/pkgs envs directories : /Users/david.hall/miniconda3/envs /Users/david.hall/.conda/envs platform : osx-64 user-agent : conda/4.5.11 requests/2.18.4 CPython/3.6.5 Darwin/17.7.0 OSX/10.13.6 UID:GID : 501:20 netrc file : None offline mode : False ```
ocefpaf commented 6 years ago

Should I be making these split packages noarch?

Not sure what you mean here.

They don't actually install any code themselves - just list the dependencies they need.

My understanding is that you need a noarch version of airflow=1.10, if that is possible, no?

halldc commented 6 years ago

Airflow itself can't be noarch because it's requirements depend on the python version. I was talking about turning the extra packages (e.g. airflow-with-celery) into noarch packages. This can be done, but it doesn't solve the issue mentioned here (extras don't work with py >27).

Instead of using an exact pinning (which includes the build variant), I should be pinning using max_pin. This stops the extra package from depending on the python version. Watch out for an incoming PR. 🎉