search

pypa / pipenv

Python Development Workflow for Humans.
https://pipenv.pypa.io
MIT License
24.87k stars 1.87k forks source link

Locking fails on wheel packages specified with platform conditionals #4106

Closed syagev closed 2 years ago

syagev commented 4 years ago

Issue description

Using latest master (https://github.com/pypa/pipenv/commit/d10b2a216a25623ba9b3e3c4ce4573e0d764c1e4) pipenv fails to lock packages specified with file= which are not on the host platform. E.g with Pipfile:

[[source]]
name = "pypi"
url = "https://pypi.org/simple"
verify_ssl = true

[packages]
torchvision_win = {os_name = "=='nt'",file = "https://download.pytorch.org/whl/cu100/torchvision-0.4.0-cp35-cp35m-win_amd64.whl"}"}

[dev-packages]

[requires]
python_version = "3.5"

Expected result

We expect pipenv install to work both on Windows and Linux. It's understood that Pipfile.lock might be different, but the locking should complete successfully.

Actual result

Running pipenv lock on Windows works, but on linux errors with:

Traceback (most recent call last):
  File "/tmp/src/pipenv/pipenv/resolver.py", line 807, in <module>
    main()
  File "/tmp/src/pipenv/pipenv/resolver.py", line 803, in main
    parsed.requirements_dir, parsed.packages, parse_only=parsed.parse_only)
  File "/tmp/src/pipenv/pipenv/resolver.py", line 785, in _main
    resolve_packages(pre, clear, verbose, system, write, requirements_dir, packages)
  File "/tmp/src/pipenv/pipenv/resolver.py", line 753, in resolve_packages
    requirements_dir=requirements_dir,
  File "/tmp/src/pipenv/pipenv/resolver.py", line 736, in resolve
    req_dir=requirements_dir
  File "/tmp/src/pipenv/pipenv/utils.py", line 1371, in resolve_deps
    req_dir=req_dir,
  File "/tmp/src/pipenv/pipenv/utils.py", line 1076, in actually_resolve_deps
    deps, index_lookup, markers_lookup, project, sources, req_dir, clear, pre
  File "/tmp/src/pipenv/pipenv/utils.py", line 643, in create
    pre=pre, clear=clear
  File "/tmp/src/pipenv/pipenv/utils.py", line 466, in get_metadata
    req, resolver=transient_resolver
  File "/tmp/src/pipenv/pipenv/utils.py", line 604, in get_deps_from_req
    best_match = pypi.find_best_match(req.ireq) if pypi else None
  File "/tmp/src/pipenv/pipenv/patched/piptools/repositories/pypi.py", line 183, in find_best_match
    matching_candidates = [candidates_by_version[ver] for ver in matching_versions]
  File "/tmp/src/pipenv/pipenv/patched/piptools/repositories/pypi.py", line 183, in <listcomp>
    matching_candidates = [candidates_by_version[ver] for ver in matching_versions]
  File "/tmp/src/pipenv/pipenv/vendor/packaging/specifiers.py", line 183, in filter
    parsed_version = self._coerce_version(version)
  File "/tmp/src/pipenv/pipenv/vendor/packaging/specifiers.py", line 133, in _coerce_version
    version = parse(version)
  File "/tmp/src/pipenv/pipenv/vendor/packaging/version.py", line 28, in parse
    return Version(version)
  File "/tmp/src/pipenv/pipenv/vendor/packaging/version.py", line 219, in __init__
    match = self._regex.search(version)
TypeError: expected string or bytes-like object

Steps to replicate

Given above

$ pipenv --support Pipenv version: `'2018.11.27.dev0'` Pipenv location: `'/tmp/src/pipenv/pipenv'` Python location: `'/usr/bin/python3.5'` Python installations found: - `3.6.7`: `/usr/bin/python3` - `3.6.7`: `/usr/bin/python3.6m` - `3.6.7`: `/usr/bin/python3.6` - `3.5.6`: `/usr/bin/python3.5` - `3.5.6`: `/usr/bin/python3.5m` PEP 508 Information: ``` {'implementation_name': 'cpython', 'implementation_version': '3.5.6', 'os_name': 'posix', 'platform_machine': 'x86_64', 'platform_python_implementation': 'CPython', 'platform_release': '4.4.0-18362-Microsoft', 'platform_system': 'Linux', 'platform_version': '#476-Microsoft Fri Nov 01 16:53:00 PST 2019', 'python_full_version': '3.5.6', 'python_version': '3.5', 'sys_platform': 'linux'} ``` System environment variables: - `LESSCLOSE` - `WSLENV` - `PWD` - `PIP_PYTHON_PATH` - `LANG` - `PYTHONFINDER_IGNORE_UNSUPPORTED` - `NAME` - `LS_COLORS` - `PIP_SHIMS_BASE_MODULE` - `USER` - `LESSOPEN` - `SHELL` - `PYTHONDONTWRITEBYTECODE` - `LOGNAME` - `HOSTTYPE` - `XDG_DATA_DIRS` - `PIP_DISABLE_PIP_VERSION_CHECK` - `HOME` - `TERM` - `_` - `SHLVL` - `LIBGL_ALWAYS_INDIRECT` - `DISPLAY` - `WSL_DISTRO_NAME` - `PATH` Pipenv–specific environment variables: Debug–specific environment variables: - `PATH`: `/home/stav/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/mnt/c/ProgramData/DockerDesktop/version-bin:/mnt/c/Program Files/Docker/Docker/Resources/bin:/mnt/c/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v10.1/bin:/mnt/c/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v10.1/libnvvp:/mnt/c/Program Files/Microsoft MPI/Bin/:/mnt/d/Dev:/mnt/c/Users/styagev/.azure-kubectl:/mnt/c/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v10.0/bin:/mnt/c/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v10.0/libnvvp:/mnt/c/Program Files (x86)/Microsoft SDKs/Azure/CLI2/wbin:/mnt/c/Program Files/Python35/Scripts/:/mnt/c/Program Files/Python35/:/mnt/c/Program Files/Microsoft HPC Pack 2016/Bin/:/mnt/c/Perl64/site/bin:/mnt/c/Perl64/bin:/mnt/c/Program Files/Python36/Scripts/:/mnt/c/Program Files/Python36/:/mnt/c/Program Files/Common Files/Microsoft Shared/Microsoft Online Services:/mnt/c/Program Files (x86)/Common Files/Microsoft Shared/Microsoft Online Services:/mnt/c/Program Files/dotnet/:/mnt/c/Program Files/MATLAB/R2019a/runtime/win64:/mnt/c/Program Files/MATLAB/R2019a/bin:/mnt/c/Program Files/Microsoft SQL Server/130/Tools/Binn/:/mnt/c/Program Files (x86)/IntelSWTools/compilers_and_libraries/windows/redist/intel64/mkl:/mnt/d/Dev/opencv-3.3.1/build/x64/vc14/bin:/mnt/c/ProgramData/chocolatey/bin:/mnt/c/Program Files/Go/bin:/mnt/c/ProgramData/chocolatey/lib/yasm/tools:/mnt/c/WINDOWS/system32:/mnt/c/WINDOWS:/mnt/c/WINDOWS/System32/Wbem:/mnt/c/WINDOWS/System32/WindowsPowerShell/v1.0/:/mnt/c/WINDOWS/System32/OpenSSH/:/mnt/d/Dev/ffmpeg-20181001-dcbd89e-win64-static/bin:/mnt/c/Program Files/Python27:/mnt/c/Program Files/Python27/Scripts:/mnt/c/Program Files/Java/jre1.8.0_201/bin:/mnt/c/Program Files (x86)/NVIDIA Corporation/PhysX/Common:/mnt/c/Program Files/Microsoft SQL Server/Client SDK/ODBC/170/Tools/Binn/:/mnt/c/Program Files/nodejs/:/mnt/c/Program Files/Intel/WiFi/bin/:/mnt/c/Program Files/Common Files/Intel/WirelessCommon/:/mnt/c/Program Files/PowerShell/6/:/mnt/c/Program Files/NVIDIA Corporation/Nsight Compute 2019.4.0/:/mnt/c/Program Files/TortoiseGit/bin:/mnt/c/Program Files (x86)/Intel/Intel(R) Management Engine Components/DAL:/mnt/c/Program Files/Intel/Intel(R) Management Engine Components/DAL:/mnt/c/Program Files/Git/cmd:/mnt/c/Users/styagev/AppData/Local/Microsoft/WindowsApps:/mnt/c/Users/styagev/go/bin:/mnt/c/Users/styagev/AppData/Roaming/Python/Python36/Scripts:/mnt/d/Dev/wxWidgets/lib/vc_dll:/mnt/c/Users/styagev/AppData/Local/Programs/Microsoft VS Code/bin:/mnt/d/Dev/gtk-build/gtk/x64/release/bin:/mnt/c/Program Files (x86)/Graphviz2.38/bin:/mnt/c/Users/styagev/AppData/Roaming/npm:/mnt/c/Program Files/Intel/WiFi/bin/:/mnt/c/Program Files/Common Files/Intel/WirelessCommon/:/snap/bin` - `SHELL`: `/bin/bash` - `LANG`: `C.UTF-8` - `PWD`: `/mnt/d/Projects/PrincetonServices/src/OfflineSystem/e2e_simulation` --------------------------- Contents of `Pipfile` ('/mnt/d/Projects/PrincetonServices/src/OfflineSystem/Pipfile'): ```toml [[source]] name = "pypi" url = "https://pypi.org/simple" verify_ssl = true [packages] attrs = "<20.0.0" azure-datalake-store = "<1.0.0" azure-mgmt-storage = "<4.0.0" azure-storage = "<1.0.0" azure-storage-blob = "<2.2.0" bokeh = "*" cntk-gpu = "==2.5.1" distributed = "<3.0.0" ffmpeg-python = "<1.0.0" filelock = "<4.0.0" future = "<1.0.0" librosa = "<0.7.0" llvmlite = "<1.0.0" matplotlib = "<4.0.0" numba = "==0.42.0" numpy = "<1.17.0" opencv-contrib-python = "<4.2.0" openpyxl = ">=2.6.0,<2.7.0" pandas = "<0.26.0" praatio = "<3.8.0" pydot-ng = "*" scipy = "<1.4.0" seaborn = "<1.0.0" sounddevice = "<0.4.0" soundfile = "<0.11.0" tensorboardx = "<2.0" tqdm = "<5.0.0" cattrs = {git = "https://github.com/Tinche/cattrs",ref = "v0.9.0"} dask = {editable = true,git = "https://github.com/syagev/dask",ref = "optional_stats_2.6.0",extras = ["dataframe", "distributed"]} pyaudio = {os_name = "=='nt'"} pythonnet = {os_name = "=='nt'"} python-snappy = {os_name = "!='nt'",version = "==0.5.4"} # snappy_win = {os_name = "=='nt'",file = "https://princetonlocal.blob.core.windows.net/redist/python_snappy-0.5.4-cp35-cp35m-win_amd64.whl"} torch = {os_name = "!='nt'", file = "https://download.pytorch.org/whl/cu100/torch-1.2.0-cp35-cp35m-manylinux1_x86_64.whl"} # torch_win = {os_name = "=='nt'",file = "https://download.pytorch.org/whl/cu100/torch-1.2.0-cp35-cp35m-win_amd64.whl"} fastparquet = {editable = true,git = "https://github.com/syagev/fastparquet",ref = "optional_stats_0.3.2"} [dev-packages] jupyter = "*" jupyter_kernel_gateway = "==2.2.0" jupytext = "*" mypy = "*" notebook = "==5.7.5" ptvsd = "*" pylint = "*" pytest = "*" pyzmq = "*" rope = "*" tensorflow = "<=2.0.0" tensorboard = "<=2.0.0" # torchvision_win = {os_name = "=='nt'",file = "https://download.pytorch.org/whl/cu100/torchvision-0.4.0-cp35-cp35m-win_amd64.whl"} line-profiler = "*" graphviz = "*" [requires] python_version = "3.5" ``` Contents of `Pipfile.lock` ('/mnt/d/Projects/PrincetonServices/src/OfflineSystem/Pipfile.lock'): ```json { "_meta": { "hash": { "sha256": "f05f9e300c88030ff492b1c6c019990ad9d8656221ef7c28e7636a2f86a79791" }, "pipfile-spec": 6, "requires": { "python_version": "3.5" }, "sources": [ { "name": "pypi", "url": "https://pypi.org/simple", "verify_ssl": true } ] }, "default": {}, "develop": {} } ```
matteius commented 2 years ago

Can this be rechecked on pipenv=2022.8.19?