Pex fails to resolve requirements, but Pip succeeds

[NiklasRosenstein]

Pex version: 2.1.136

I fear this example might eventually not be reproducible when new versions of things get released, but today it fails.

$ pex novella==0.2.6 mkdocs mkdocs-material pydoc-markdown==4.8.2 -c novella
Failed to resolve compatible distributions:
1: databind.core==4.4.2 requires typing-extensions<4.7,>=3.10.0 but typing_extensions 4.9.0 was resolved
2: databind.json==4.4.2 requires typing-extensions<4.7,>=3.10.0 but typing_extensions 4.9.0 was resolved

$ python -m venv venv
$ venv/bin/pip install novella==0.2.6 mkdocs mkdocs-material pydoc-markdown==4.8.2
[...]
Successfully installed Deprecated-1.2.14 PyYAML-6.0.1 astor-0.8.1 babel-2.14.0 black-23.12.1 certifi-2023.11.17 charset-normalizer-3.3.2 click-8.1.7 colorama-0.4.6 craftr-dsl-0.7.7 databind.core-4.4.2 databind.json-4.4.2 docspec-2.2.1 docspec-python-2.2.1 docstring-parser-0.11 ghp-import-2.1.0 idna-3.6 importlib-metadata-7.0.1 jinja2-3.1.3 markdown-3.5.2 markupsafe-2.1.3 mergedeep-1.3.4 mkdocs-1.5.3 mkdocs-material-9.5.3 mkdocs-material-extensions-1.3.1 mypy-extensions-1.0.0 novella-0.2.6 nr-date-2.1.0 nr-stream-1.1.5 nr.util-0.8.12 packaging-23.2 paginate-0.5.6 pathspec-0.12.1 platformdirs-4.1.0 pydoc-markdown-4.8.2 pygments-2.17.2 pymdown-extensions-10.7 python-dateutil-2.8.2 pyyaml-env-tag-0.1 regex-2023.12.25 requests-2.31.0 six-1.16.0 termcolor-1.1.0 tomli-2.0.1 tomli_w-1.0.0 typeapi-2.1.1 typing-extensions-4.6.3 urllib3-2.1.0 watchdog-2.3.1 wrapt-1.16.0 yapf-0.40.2 zipp-3.17.0

Even pinning the exact versions of mkdocs and mkdocs-material doesn't fix it.

$ pex novella==0.2.6 mkdocs==1.5.3 mkdocs-material==9.5.3 pydoc-markdown==4.8.2 -c novella -v
[same error]

Pinning typing-extensions==4.6.3 then finally solves it.

[jsirois]

@NiklasRosenstein modern Pex supports many --pip-version values. Perhaps try --pip-version latest.

The key concept with Pex to keep in mind is I never break existing users. That means, as I add features, the old defaults remain the same as they did in the past; so sometimes you need to use --help to discover these new, non-default, features. In this particular case, the default Pip is ~3 years old, 20.3.4.

[jsirois]

It looks like your version supports Pip 23.1.2: https://github.com/pantsbuild/pex/blob/b17ab8ebb667a940da999f592a87a4c76d3dc1ea/pex/pip/version.py#L141-L149

If you upgrade to newer versions of Pex you get 23.2+ support which also means Python 3.12 support (up through the very latest Pip release in fact).

[jsirois]

@NiklasRosenstein what version of python are you using here? It's hard to repro without that key detail.

[jsirois]

Ok, here's what I find guessing Python 3.8:

$ pex -V
2.1.156
$ pex --python python3.8 novella==0.2.6 mkdocs mkdocs-material pydoc-markdown==4.8.2 -c novella --pip-version vendored
Failed to resolve compatible distributions:
1: databind.core==4.4.2 requires typing-extensions<4.7,>=3.10.0 but typing_extensions 4.9.0 was resolved
2: databind.json==4.4.2 requires typing-extensions<4.7,>=3.10.0 but typing_extensions 4.9.0 was resolved
$ pex --python python3.8 novella==0.2.6 mkdocs mkdocs-material pydoc-markdown==4.8.2 -c novella --pip-version 23.1.2
Failed to resolve compatible distributions:
1: databind.core==4.4.2 requires typing-extensions<4.7,>=3.10.0 but typing_extensions 4.9.0 was resolved
2: databind.json==4.4.2 requires typing-extensions<4.7,>=3.10.0 but typing_extensions 4.9.0 was resolved
$ pex --python python3.8 novella==0.2.6 mkdocs mkdocs-material pydoc-markdown==4.8.2 -c novella --pip-version 23.2
Traceback (most recent call last):
  File "/home/jsirois/.pyenv/versions/3.8.18/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/home/jsirois/.pyenv/versions/3.8.18/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/tmp/tmptksuwj39/__main__.py", line 106, in <module>
    bootstrap_pex(__entry_point__, execute=__execute__, venv_dir=__venv_dir__)
  File "/tmp/tmptksuwj39/.bootstrap/pex/pex_bootstrapper.py", line 627, in bootstrap_pex
    pex.PEX(entry_point).execute()
  File "/tmp/tmptksuwj39/.bootstrap/pex/pex.py", line 561, in execute
    sys.exit(self._wrap_coverage(self._wrap_profiling, self._execute))
  File "/tmp/tmptksuwj39/.bootstrap/pex/pex.py", line 468, in _wrap_coverage
    return runner(*args)
  File "/tmp/tmptksuwj39/.bootstrap/pex/pex.py", line 499, in _wrap_profiling
    return runner(*args)
  File "/tmp/tmptksuwj39/.bootstrap/pex/pex.py", line 605, in _execute
    return self.execute_entry(
  File "/tmp/tmptksuwj39/.bootstrap/pex/pex.py", line 774, in execute_entry
    return self.execute_entry_point(entry_point)
  File "/tmp/tmptksuwj39/.bootstrap/pex/pex.py", line 792, in execute_entry_point
    return runner()
  File "/home/jsirois/.pex/installed_wheels/bbc350f9f1db71ebbc2a08cc3e4674a96af20a54fffb098634b4ddda2c5f1230/novella-0.2.6-py3-none-any.whl/novella/__main__.py", line 184, in main
    raise exception
  File "/home/jsirois/.pex/installed_wheels/bbc350f9f1db71ebbc2a08cc3e4674a96af20a54fffb098634b4ddda2c5f1230/novella-0.2.6-py3-none-any.whl/novella/__main__.py", line 172, in main
    context = novella.execute_file(args.config_file, code)
  File "/home/jsirois/.pex/installed_wheels/bbc350f9f1db71ebbc2a08cc3e4674a96af20a54fffb098634b4ddda2c5f1230/novella-0.2.6-py3-none-any.whl/novella/novella.py", line 47, in execute_file
    Closure(None, None, context).run_code(code if code is not None else file.read_text(), str(file))
  File "/home/jsirois/.pyenv/versions/3.8.18/lib/python3.8/pathlib.py", line 1236, in read_text
    with self.open(mode='r', encoding=encoding, errors=errors) as f:
  File "/home/jsirois/.pyenv/versions/3.8.18/lib/python3.8/pathlib.py", line 1222, in open
    return io.open(self, mode, buffering, encoding, errors, newline,
  File "/home/jsirois/.pyenv/versions/3.8.18/lib/python3.8/pathlib.py", line 1078, in _opener
    return self._accessor.open(self, flags, mode)
FileNotFoundError: [Errno 2] No such file or directory: 'build.novella'

So, you need at least Pip 23.2 to get a successful resolve and it appears novella is broken.

[jsirois]

@NiklasRosenstein actually your current Pex works, but you need to specify 2 non-default options --pip-version latest --resolver-version pip-2020-resolver:

$ pex --python python3.8 pex==2.1.136 -cpex -- novella==0.2.6 mkdocs mkdocs-material pydoc-markdown==4.8.2 -c novella --pip-version latest --resolver-version pip-2020-resolver
...
  File "/home/jsirois/.pex/installed_wheels/bbc350f9f1db71ebbc2a08cc3e4674a96af20a54fffb098634b4ddda2c5f1230/novella-0.2.6-py3-none-any.whl/novella/__main__.py", line 184, in main
    raise exception
  File "/home/jsirois/.pex/installed_wheels/bbc350f9f1db71ebbc2a08cc3e4674a96af20a54fffb098634b4ddda2c5f1230/novella-0.2.6-py3-none-any.whl/novella/__main__.py", line 172, in main
    context = novella.execute_file(args.config_file, code)
  File "/home/jsirois/.pex/installed_wheels/bbc350f9f1db71ebbc2a08cc3e4674a96af20a54fffb098634b4ddda2c5f1230/novella-0.2.6-py3-none-any.whl/novella/novella.py", line 47, in execute_file
    Closure(None, None, context).run_code(code if code is not None else file.read_text(), str(file))
  File "/home/jsirois/.pyenv/versions/3.8.18/lib/python3.8/pathlib.py", line 1236, in read_text
    with self.open(mode='r', encoding=encoding, errors=errors) as f:
  File "/home/jsirois/.pyenv/versions/3.8.18/lib/python3.8/pathlib.py", line 1222, in open
    return io.open(self, mode, buffering, encoding, errors, newline,
  File "/home/jsirois/.pyenv/versions/3.8.18/lib/python3.8/pathlib.py", line 1078, in _opener
    return self._accessor.open(self, flags, mode)
FileNotFoundError: [Errno 2] No such file or directory: 'build.novella'

At the --pip-version 23.2 cutoff, Pip dropped support for their legacy resolver; so you no longer need to tell Pex --resolver-version pip-2020-resolver to move away from the old default of --resolver-version pip-legacy-resolver (Pip maintained both resolvers internally for some time during their transition; so Pex did as well, defaulting to the original legacy as long as it could).

[jsirois]

@NiklasRosenstein I've labelled this a question and I'd like to close as answered, since I think using --pip-version and --resolver-version solves your problem. Please let me know one way or the other if this solves your problem.

[jsirois]

Aha, novella is not broken, it just expects a novella.build file in CWD. A better example:

$ pex --python python3.8 pex==2.1.136 -cpex -- novella==0.2.6 mkdocs mkdocs-material pydoc-markdown==4.8.2 -c novella --pip-version latest --resolver-version pip-2020-resolver -- --version
0.2.6

[NiklasRosenstein]

Thanks for the detailed response @jsirois! Indeed it works with --pip-version latest --resolver-version pip-2020-resolver. It takes quite a while longer to build (130s vs. 50s) but I suppose that's the trade off of relying on resolution and not using exact constraints. Probably the would-be conflict takes extra time to resolve

Edit: Running the second time it seems we're back to 50s