package use specified marker in Pipfile may generates error logical condition in Pipfile.lock

leileigong commented 5 years ago

Issue description

i wanna install py2exe via pipenv. so i write "py2exe-py2" = {version = "==0.6.9", markers = "python_version <= '3.4'", sys_platform = "== 'win32'"} to my Pipfile, this will result in this bug.

but if write "py2exe-py2" = {version = "==0.6.9", markers = "python_version <= '3.4' and sys_platform == 'win32'"} to my Pipfile, this will work fine.

my virtual python version is 3.6

Expected result

Pipfile.lock content should be:

        "py2exe-py2": {
            "hashes": [
                "sha256:c66c73aaca3e68ef52dcfcd6222d83d3622a50a7d0bd3e762814464da51d8371"
            ],
            "index": "pypi",
            "markers": "python_version <= '3.4' and sys_platform == 'win32'",
            "version": "==0.6.9"
        },

the shell should output:

Ignoring py2exe-py2: markers 'python_version <= "3.4" and sys_platform == "win32"' don't match your environment

Actual result

1.Pipfile.lock content actural be:

        "py2exe-py2": {
            "hashes": [
                "sha256:c66c73aaca3e68ef52dcfcd6222d83d3622a50a7d0bd3e762814464da51d8371"
            ],
            "index": "pypi",
            "markers": "python_version <= '3.4' or sys_platform == 'win32'",
            "version": "==0.6.9"
        },

2.the shell actural output:

An error occurred while installing py2exe-py2==0.6.9 ; python_version <= '3.4' or sys_platform == 'win32' --hash=sha256:c66c73aaca3e68ef52dcfcd6222d83d3622a50a7d0bd3e762814464da51d8371! Will try again.
....
Installing initially failed dependencies…
     ================================ 2/2 - 00:00:07
[pipenv.exceptions.InstallError]:   File "c:\python36\lib\site-packages\pipenv\cli\command.py", line 254, in install
[pipenv.exceptions.InstallError]:       editable_packages=state.installstate.editables,
[pipenv.exceptions.InstallError]:   File "c:\python36\lib\site-packages\pipenv\core.py", line 1874, in do_install
[pipenv.exceptions.InstallError]:       keep_outdated=keep_outdated
[pipenv.exceptions.InstallError]:   File "c:\python36\lib\site-packages\pipenv\core.py", line 1253, in do_init
[pipenv.exceptions.InstallError]:       pypi_mirror=pypi_mirror,
[pipenv.exceptions.InstallError]:   File "c:\python36\lib\site-packages\pipenv\core.py", line 862, in do_install_dependencies
[pipenv.exceptions.InstallError]:       _cleanup_procs(procs, False, failed_deps_queue, retry=False)
[pipenv.exceptions.InstallError]:   File "c:\python36\lib\site-packages\pipenv\core.py", line 681, in _cleanup_procs
[pipenv.exceptions.InstallError]:       raise exceptions.InstallError(c.dep.name, extra=err_lines)
[pipenv.exceptions.InstallError]: ['Collecting py2exe-py2==0.6.9 (from -r C:\\Users\\lei\\AppData\\Local\\Temp\\pipenv-m_0kxixe-requirements\\pipenv-cgoiw517-requirement.txt (line 1))']
[pipenv.exceptions.InstallError]: ['Could not find a version that satisfies the requirement py2exe-py2==0.6.9 (from -r C:\\Users\\lei\\AppData\\Local\\Temp\\pipenv-m_0kxixe-requirements\\pipenv-cgoiw517-requirement.txt (line 1)) (from versions: )', 'No matching distribution found for py2exe-py2==0.6.9 (from -r C:\\Users\\lei\\AppData\\Local\\Temp\\pipenv-m_0kxixe-requirements\\pipenv-cgoiw517-requirement.txt (line 1))']
ERROR: ERROR: Package installation failed...

Steps to replicate

add an package item to pipfile:

"py2exe-py2" = {version = "==0.6.9", markers = "python_version <= '3.4'", sys_platform = "== 'win32'"}

exec "pipenv install"

frostming commented 5 years ago

Yeah, we should check the markers before actually searching PyPI.

leileigong commented 5 years ago

i mean markers = "python_version <= '3.4'", sys_platform = "== 'win32'" in Pipfile is expressed as "markers": "python_version <= '3.4' or sys_platform == 'win32'" in Pipfile.lock, is this correct?

frostming commented 5 years ago

Oh, yeah, it comes from https://github.com/pypa/pipenv/blob/d5e446ee7bd0e769c385331714c8616c6b6738b8/pipenv/utils.py#L1611-L1614 But I dunno whether or is intended /cc @techalchemy

techalchemy commented 5 years ago

It's sort of correct, it's intended to handle the normal use case of multiple possible python versions listed. If you want to use multiple types of marker (e.g. python_version and sys_platform) you need to use standard syntax and only supply markers as a key (merging markers is incredibly hard, we've spent like months or longer on this problem in passa):

markers = "python_version <= '3.4' and sys_platform == 'win32'"

There is something we can do here but it is super difficult to test edge cases etc, so the safest answer is to just write it out

leileigong commented 5 years ago

I got another problem about marker. Pipfile content: pyscard-cp27-win32 = {path = "wheelhouse/pyscard-1.9.7-cp27-cp27m-win32.whl",markers = "python_version == '2.7'",sys_platform = "== 'win32'"} pyscard-cp36-win32 = {path = "wheelhouse/pyscard-1.9.7-cp36-cp36m-win_amd64.whl",markers = "python_version == '3.6'",sys_platform = "== 'win32'

auto generate Pipfile.lock content, list three key map "pyscard", "pyscard-cp27-win32", "pyscard-cp36-win32": "pyscard": { "hashes": [ "sha256:412c74c83e7401566e9d3d7b8b5ca965e74582a1f33179b3c1fabf1da73ebf80", "sha256:c63a87e4e7c87ce4c1299a1d8e0cae4e43f27451ca210f2c54f2dcd7467565c5" ], "markers": "python_version == '3.6' and sys_platform == 'win32'", "version": "==1.9.7" }, "pyscard-cp27-win32": { "markers": "python_version == '2.7'", "path": "wheelhouse/pyscard-1.9.7-cp27-cp27m-win32.whl", "sys_platform": "== 'win32'" }, "pyscard-cp36-win32": { "markers": "python_version == '3.6'", "path": "wheelhouse/pyscard-1.9.7-cp36-cp36m-win_amd64.whl", "sys_platform": "== 'win32'" },

So in the "pyscard" key-map. "markers": "python_version == '3.6' and sys_platform == 'win32'", Does "marker" should exist? if should , `python_version == '3.6' is correct?

techalchemy commented 5 years ago

markers don’t work on local paths as far as I understand

techalchemy commented 5 years ago

(Wrong button)

leileigong commented 5 years ago

infact, markers does work on local paths.

let's say, on python2.7 virtual env, installing pyscard from local wheel, the console output:

Ignoring pyscard: markers 'python_version == "3.6" and sys_platform == "win32"' don't match your environment
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7.
Ignoring pyscard: markers 'python_version == "3.6" and sys_platform == "win32"' don't match your environment

frostming commented 5 years ago

If I understand correctly, the resolver will add the hashes of local wheels at the end, but it can't find a key pyscard, so it creates one. The two hashes should come from the two wheels.

pypa / pipenv