Experiment: Allow using pip 23.1.x with Python 3.12

[huonw]

This adjusts the Python version upper bounds for the pip 23.1.x series, since they seem to work with Python 3.12 locally, but this was excluded by the range.

In #2172, support for Python 3.12 was added to pex. At that time, it seemed like pip 23.1.x didn't work with Python 3.12, but, some naive tests suggests it does, now:

python3.12 -m venv /tmp/venv-3.12
. /tmp/venv-3.12/bin/activate

python -m pip install -U pip==23.1 setuptools==67.6.1 wheel==0.40.0
python -m pip --version # pip 23.1 from /private/tmp/v312/lib/python3.12/site-packages/pip (python 3.12)

# example:
python -m pip install pandas

Does pex's CI agree? Is there background to the ranges chosen #2172 that isn't recorded somewhere I could find?

[jsirois]

CI gave you at least 1 answer, pkgutil.ImpImporter is gone in 3.12 and that Pip needs it: https://docs.python.org/3.11/library/pkgutil.html#pkgutil.ImpImporter

@huonw perhaps a good question is why you're trying to search for a mistake here. Is there some need driving wanting older Pip to work with 3.12? I could've messed up but I also sunk alot of time into this transition to be ready on time for Pants, etc since I initially thought it would require cutting Pex 3 / dropping Python 2.7 support.

[huonw]

Brief reply to start with because I'm on mobile: I'm investigating behaviour here because pip 23.1 is apparently working for me on arm macOS with Python 3.12, as you can see in the description (and, for instance, the same steps with an earlier version do not work, and fail with errors similar to CI here).

I'm not looking for a mistake (I apologise if it came across that way) or demanding more of your time. In fact, it's an attempt at the exact opposite: I'm doing the debugging/experimenting myself so I'm more informed before asking you any questions/getting reviews etc.

[jsirois]

Ok. Well, perhaps most simply, your quick check used pip install. There are 3 types to check there: wheel, sdist PEP-517/518 & sdist setup.py. More direct though is pip debug, which Pex uses to implement --platform:

(venv-3.12) jsirois@siroisdesign:~$ pip debug
WARNING: This command is only meant for debugging. Do not use this with automation for parsing and getting these details, since the output and options of this command may change without notice.
pip version: pip 23.1 from /tmp/venv-3.12/lib/python3.12/site-packages/pip (python 3.12)
sys.version: 3.12.1 (main, Jan  1 2024, 16:01:25) [GCC 9.4.0]
sys.executable: /tmp/venv-3.12/bin/python
sys.getdefaultencoding: utf-8
sys.getfilesystemencoding: utf-8
locale.getpreferredencoding: UTF-8
sys.platform: linux
sys.implementation:
  name: cpython
'cert' config value: Not specified
REQUESTS_CA_BUNDLE: None
CURL_CA_BUNDLE: None
pip._vendor.certifi.where(): /tmp/venv-3.12/lib/python3.12/site-packages/pip/_vendor/certifi/cacert.pem
pip._vendor.DEBUNDLED: False
vendored library versions:
  CacheControl==0.12.11
  colorama==0.4.6
  distlib==0.3.6
  distro==1.8.0
  msgpack==1.0.5
  packaging==21.3
  platformdirs==3.2.0
  pyparsing==3.0.9
  pyproject-hooks==1.0.0
  requests==2.28.2
  certifi==2022.12.07
  chardet==5.1.0
  idna==3.4
  urllib3==1.26.15
  rich==13.3.3 (Unable to locate actual module version, using vendor.txt specified version)
  pygments==2.14.0
ERROR: Exception:
Traceback (most recent call last):
  File "/tmp/venv-3.12/lib/python3.12/site-packages/pip/_internal/cli/base_command.py", line 169, in exc_logging_wrapper
    status = run_func(*args)
             ^^^^^^^^^^^^^^^
  File "/tmp/venv-3.12/lib/python3.12/site-packages/pip/_internal/commands/debug.py", line 195, in run
    show_vendor_versions()
  File "/tmp/venv-3.12/lib/python3.12/site-packages/pip/_internal/commands/debug.py", line 101, in show_vendor_versions
    show_actual_vendor_versions(vendor_txt_versions)
  File "/tmp/venv-3.12/lib/python3.12/site-packages/pip/_internal/commands/debug.py", line 81, in show_actual_vendor_versions
    actual_version = get_vendor_version_from_module(module_name)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/venv-3.12/lib/python3.12/site-packages/pip/_internal/commands/debug.py", line 61, in get_vendor_version_from_module
    module = get_module_from_module_name(module_name)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/venv-3.12/lib/python3.12/site-packages/pip/_internal/commands/debug.py", line 56, in get_module_from_module_name
    __import__(f"pip._vendor.{module_name}", globals(), locals(), level=0)
  File "/tmp/venv-3.12/lib/python3.12/site-packages/pip/_vendor/typing_extensions.py", line 1174, in <module>
    class TypeVar(typing.TypeVar, _DefaultMixin, _root=True):
TypeError: type 'typing.TypeVar' is not an acceptable base type
(venv-3.12) jsirois@siroisdesign:~$

See: https://github.com/pypa/pip/issues/12053

For completeness, when trying to get a gut feel for this sort of thing, know Pex actually uses all of pip {download,wheel,install,debug} in one form or another (at least historically, pip install use just got dropped in the last few releases).

[huonw]

Okay, thanks for the fish.

For completeness, when trying to get a gut feel for this sort of thing, know Pex actually uses all of pip {download,wheel,install,debug} in one form or another (at least historically, pip install use just got dropped in the last few releases).

Right, that's helpful gut feel.