search

pex-tool / pex

A tool for generating .pex (Python EXecutable) files, lock files and venvs.
https://docs.pex-tool.org/
Apache License 2.0
2.67k stars 263 forks source link

Dependency resolver bug #371

Closed ruthgrace closed 3 months ago

ruthgrace commented 7 years ago

I apologize -- I don't have a minimal reproducible test case for this.

I have a repo with 3rdparty/python/requirements.txt In our pants.ini we point to both our internal pypi repo with our custom modules, and a mirror of the public pypi repo.

When I ran all the tests I got

Exception message: Could not satisfy all requirements for oauthlib>=0.6.2:
    oauthlib>=0.6.2(from: Flask-OAuthlib==0.9.3), oauthlib==0.6.3(from: auth-client==0.2.3)

Which doesn't make any sense, because oauthlib>=0.6.2 and oauthlib==0.6.3 should be perfectly compatible. Flask-OAuthlib is something we're getting from our pypi mirror, whereas auth-client is from our internal custom pypi repo. oauthlib is from the pypi mirror.

I found the package that depended on auth-client, and explicitly added 3rdparty/python:oauthlib to the dependencies list in the BUILD file as suggested by @kwlzn, and this error was fixed.

jsirois commented 3 months ago

With modern Pex, using --pip-version to have Pex use newer Pips than its vendored Pip 20.3.4 in legacy mode seems to do the trick. As does just switching the vendored Pip to --resolver-version pip-2020-resolver.

Here's a session showing that, using the always backwards compatible command line defaults still errors, but using one of the two suggestions fixes:

# Pex defaults to its vendored Pip 20.3.4 resolver and its default legacy resolver mode, which fails:
:; pex "oauthlib>=0.6.2" "oauthlib==0.6.3"
pid 709267 -> /home/jsirois/.pex/venvs/960deab6b1961767adc5420cc4589b505810b49d/bcd5cfc9f138f1afceeb38adfb3b9b1c5b07dbe9/bin/python -sE /home/jsirois/.pex/venvs/960deab6b1961767adc5420cc4589b505810b49d/bcd5cfc9f138f1afceeb38adfb3b9b1c5b07dbe9/pex --disable-pip-version-check --no-python-version-warning --exists-action a --no-input --use-deprecated legacy-resolver --isolated -q --cache-dir /home/jsirois/.pex/pip/20.3.4-patched/pip_cache --log /tmp/pex-pip-log.xygkhfly/pip.log download --dest /home/jsirois/.pex/downloads/resolver_download._2e9amm7/home.jsirois..local.bin.tools.venv.bin.python oauthlib>=0.6.2 oauthlib==0.6.3 --retries 5 --timeout 15 exited with 1 and STDERR:
pip: ERROR: Double requirement given: oauthlib==0.6.3 (already in oauthlib>=0.6.2, name='oauthlib')

# Pex's vendored Pip is 20.3.4 which defaults to the legacy resolver. You can switch that though with --resolver-version pip-2020-resolver` and things work:
:; pex --resolver-version pip-2020-resolver "oauthlib>=0.6.2" "oauthlib==0.6.3"
Pex 2.16.0 ephemeral hermetic environment with 2 requirements and 1 activated distribution.
Python 3.11.9 (main, Apr 26 2024, 19:20:24) [GCC 13.2.0] on linux
Type "help", "pex", "copyright", "credits" or "license" for more information.
>>> pex()
Running from ephemeral loose PEX directory: /tmp/tmp102zc2gs
Requirements:
  oauthlib==0.6.3
  oauthlib>=0.6.2
Activated Distributions:
  oauthlib-0.6.3-py3-none-any.whl
>>>

# Modern Pip (this picks 24.2 today) uses the pip-2020-resolver by default; so this works:
:; pex --pip-version latest "oauthlib>=0.6.2" "oauthlib==0.6.3"
Pex 2.16.0 ephemeral hermetic environment with 2 requirements and 1 activated distribution.
Python 3.11.9 (main, Apr 26 2024, 19:20:24) [GCC 13.2.0] on linux
Type "help", "pex", "copyright", "credits" or "license" for more information.
>>> pex()
Running from ephemeral loose PEX directory: /tmp/tmpxx932_tm
Requirements:
  oauthlib==0.6.3
  oauthlib>=0.6.2
Activated Distributions:
  oauthlib-0.6.3-py3-none-any.whl
>>>