search

holoviz-dev / pyctdev

Python packaging Common Tasks for Developers
BSD 3-Clause "New" or "Revised" License
10 stars 2 forks source link

Version exclusion != not respected on Windows #90

Open maximlt opened 2 years ago

maximlt commented 2 years ago

Datashader's CI is failing on Windows. Its setup.py declares numpy >=1.7,!=1.22 and for some (still unknown, at least to me) reason the exclamation mark isn't passed down to conda which then translates it as numpy=1.22, exactly what we don't want!

See https://github.com/holoviz/datashader/runs/5788134696?check_suite_focus=true#step:8:149 where this commands:

conda install -y   "dask >=0.18.0" "datashape >=0.5.1" "numba >=0.51" "numpy >=1.7,!=1.22" "pandas >=0.24.1" "pillow >=3.1.1" "xarray >=0.9.6" "colorcet >=0.9.0" "param >=1.6.1" "pyct >=0.4.5" "scipy" "pytest >=3.9.3" "pytest-benchmark >=3.0.0" "pytest-cov" "codecov" "flake8" "nbconvert" "nbsmoke >0.5" "fastparquet >=0.1.6" "holoviews >=1.10.0" "bokeh" "pyarrow" "netcdf4" "twine" "rfc3986" "keyring" "spatialpandas" "rioxarray" "rasterio"

leads to:

    - numpy[version='>=1.7,=1.22']

In this PR https://github.com/pyviz-dev/pyctdev/pull/87 I tried to fix this issue by setting shell=False to CmdAction in a couple of occurrences, which failed miserably!

hoxbro commented 2 years ago

Could this be fixed by #73?

maximlt commented 2 years ago

Oh well spotted! @philippjfr do you by any chance remember why you opened #73?

philippjfr commented 2 years ago

Ha no, 2 years is a long time. Presumably for the same reason though.

maximlt commented 2 years ago

Ok thanks anyway! Looks like a good candidate for fixing this issue, I'll have a go at it.

maximlt commented 2 years ago

The root cause of this issue seems to be a conda bug (https://github.com/conda/conda/issues/10522) by which a conda command executed on Windows with cmd.exe doesn't handle well characters like !. pyctdev uses doit which uses subprocess.Popen to run conda commands like conda install .... On Windows subprocess.Popen happens to run that command with cmd.exe when shell=True, which is the default of doit. Hence, the bug observed with pyctdev.