search

pypa / pip

The Python package installer
https://pip.pypa.io/
MIT License
9.51k stars 3.02k forks source link

Pip (seemingly) ignores requires-python on package #11046

Closed jtratner closed 2 years ago

jtratner commented 2 years ago

Description

Typing-extensions released a new version yesterday that broke compatibility with python3.6 (makes sense!); however, now we are hitting issues that pip 21.3.1 in a python3.6 environment does not correctly skip typing-extensions 4.2.0.

The weird part about the whole thing is that the package metadata for typing-extensions shows a Requires-Python: >=3.7 when downloaded, but somehow pip doesn't see it? (Or perhaps it was uploaded incorrectly to pypi).

Totally understand we're using a very out of date version of python, but that's where we're at right now. Really appreciate any help debugging this and happy to move this to somewhere else if there's a better place to put it.

Some additional background - the tarball clearly has different requires python

›› cat ~/Downloads/typing_extensions-4.2.0/PKG-INFO
Metadata-Version: 2.1
Name: typing_extensions
Version: 4.2.0
Summary: Backported and Experimental Type Hints for Python 3.7+
Keywords: annotations,backport,checker,checking,function,hinting,hints,type,typechecking,typehinting,typehints,typing
Author-email: "Guido van Rossum, Jukka Lehtosalo, Łukasz Langa, Michael Lee" <levkivskyi@gmail.com>
Requires-Python: >=3.7

Plus HTML on PyPI shows that it has the right requires-python attribute in the HTML.

163880604-ef9d3091-7dc3-4e05-b696-6f03593ac700

`

Expected behavior

Installing on python3.6 skips typing-extensions 4.2.0 and instead installs 4.1.1

pip version

21.3.1

Python version

3.6.11

OS

Mac 10.15 (reproduces on Ubuntu as well)

How to Reproduce

virtualenv --python=python3.6 somevenv source somevenv/bin/activate python --version pip --version pip install -v --no-cache-dir typing-extensions # should not be 4.2

Output

$ virtualenv --python=python3.6 somevenv

created virtual environment CPython3.6.11.final.0-64 in 3895ms
  creator CPython3Posix(dest=/Users/jtratner/somevenv, clear=False, no_vcs_ignore=False, global=False)
  seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/Users/jtratner/Library/Application Support/virtualenv)
    added seed packages: pip==21.3.1, setuptools==59.6.0, wheel==0.37.1
  activators BashActivator,CShellActivator,FishActivator,PowerShellActivator,PythonActivator,XonshActivator

$ source somevenv/bin/activate

$ pip --version
pip 21.3.1 from /Users/jtratner/somevenv/lib/python3.6/site-packages/pip (python 3.6)

$ pip install -v --no-cache-dir typing-extensions
Using pip 21.3.1 from /Users/jtratner/somevenv/lib/python3.6/site-packages/pip (python 3.6)
Collecting typing-extensions
  Downloading typing_extensions-4.2.0-py3-none-any.whl (24 kB)
Installing collected packages: typing-extensions
Successfully installed typing-extensions-4.2.0
  Link requires a different Python (3.6.11 not in: '>=3.7'): https://files.pythonhosted.org/packages/9f/8b/a094f5da22d7abf5098205367b3296dd15b914f4232af5ca39ba6214d08c/pip-22.0-py3-none-any.whl#sha256=6cb1ea2bd7fda0668e26ae8c3e45188f301a7ef17ff22efe1f70f3643e56a822 (from https://pypi.org/simple/pip/) (requires-python:>=3.7)
  Link requires a different Python (3.6.11 not in: '>=3.7'): https://files.pythonhosted.org/packages/4a/ca/e72b3b399d7a8cb34311aa8f52924108591c013b09f0268820afb4cd96fb/pip-22.0.tar.gz#sha256=d3fa5c3e42b33de52bddce89de40268c9a263cd6ef7c94c40774808dafb32c82 (from https://pypi.org/simple/pip/) (requires-python:>=3.7)
  Link requires a different Python (3.6.11 not in: '>=3.7'): https://files.pythonhosted.org/packages/89/a1/2f4e58eda11e591fbfa518233378835679fc5ab766b690b3df85215014d5/pip-22.0.1-py3-none-any.whl#sha256=30739ac5fb973cfa4399b0afff0523d4fe6bed2f7a5229333f64d9c2ce0d1933 (from https://pypi.org/simple/pip/) (requires-python:>=3.7)
  Link requires a different Python (3.6.11 not in: '>=3.7'): https://files.pythonhosted.org/packages/63/71/5686e51f06fa59da55f7e81c3101844e57434a30f4a0d7456674d1459841/pip-22.0.1.tar.gz#sha256=7fd7a92f2fb1d2ac2ae8c72fb10b1e640560a0361ed4427453509e2bcc18605b (from https://pypi.org/simple/pip/) (requires-python:>=3.7)
  Link requires a different Python (3.6.11 not in: '>=3.7'): https://files.pythonhosted.org/packages/83/b5/df8640236faa5a3cb80bfafd68e9fb4b22578208b8398c032ccff803f9e0/pip-22.0.2-py3-none-any.whl#sha256=682eabc4716bfce606aca8dab488e9c7b58b0737e9001004eb858cdafcd8dbdd (from https://pypi.org/simple/pip/) (requires-python:>=3.7)
  Link requires a different Python (3.6.11 not in: '>=3.7'): https://files.pythonhosted.org/packages/d9/c1/146b24a7648fdf3f8b4dc6521ab0b26ac151ef903bac0b63a4e1450cb4d1/pip-22.0.2.tar.gz#sha256=27b4b70c34ec35f77947f777070d8331adbb1e444842e98e7150c288dc0caea4 (from https://pypi.org/simple/pip/) (requires-python:>=3.7)
  Link requires a different Python (3.6.11 not in: '>=3.7'): https://files.pythonhosted.org/packages/6a/df/a6ef77a6574781a668791419ffe366c8acd1c3cf4709d210cb53cd5ce1c2/pip-22.0.3-py3-none-any.whl#sha256=c146f331f0805c77017c6bb9740cec4a49a0d4582d0c3cc8244b057f83eca359 (from https://pypi.org/simple/pip/) (requires-python:>=3.7)
  Link requires a different Python (3.6.11 not in: '>=3.7'): https://files.pythonhosted.org/packages/88/d9/761f0b1e0551a3559afe4d34bd9bf68fc8de3292363b3775dda39b62ce84/pip-22.0.3.tar.gz#sha256=f29d589df8c8ab99c060e68ad294c4a9ed896624f6368c5349d70aa581b333d0 (from https://pypi.org/simple/pip/) (requires-python:>=3.7)
  Link requires a different Python (3.6.11 not in: '>=3.7'): https://files.pythonhosted.org/packages/4d/16/0a14ca596f30316efd412a60bdfac02a7259bf8673d4d917dc60b9a21812/pip-22.0.4-py3-none-any.whl#sha256=c6aca0f2f081363f689f041d90dab2a07a9a07fb840284db2218117a52da800b (from https://pypi.org/simple/pip/) (requires-python:>=3.7)
  Link requires a different Python (3.6.11 not in: '>=3.7'): https://files.pythonhosted.org/packages/33/c9/e2164122d365d8f823213a53970fa3005eb16218edcfc56ca24cb6deba2b/pip-22.0.4.tar.gz#sha256=b3a9de2c6ef801e9247d1527a4b16f92f2cc141cd1489f3fffaf6a9e96729764 (from https://pypi.org/simple/pip/) (requires-python:>=3.7)

Code of Conduct

jtratner commented 2 years ago

If helpful, typing-extensions uses twine to do uploads (from discussion in python/typing_extensions#14 ) and their pyproject.toml looks like it has the correct format of Requires-Python: as well.

jtratner commented 2 years ago

(I'm noticing now that pip has now released v22 and dropped support for python3.6 - so no worries if you don't have bandwidth to address it! Just hoping for short-term fix until we can upgrade away from python3.6)

uranusjr commented 2 years ago

That’s weird. Pip itself is released with requires-python metadata, and from your -v output pip can actually correctly identify those on PyPI (I think those are emitted during the post-invocation version-check code). But I can’t tell why it can’t pick up those on the typing-extensions page, they look exactly the same from what I tell…

<a href="https://files.pythonhosted.org/packages/75/e1/932e06004039dd670c9d5e1df0cd606bf46e29a28e65d5bb28e894ea29c9/typing_extensions-4.2.0-py3-none-any.whl#sha256=6657594ee297170d19f67d55c05852a874e7eb634f4f753dbd667855e07c1708" data-requires-python="&gt;=3.7">typing_extensions-4.2.0-py3-none-any.whl</a><br/>
<a href="https://files.pythonhosted.org/packages/fe/71/1df93bd59163c8084d812d166c907639646e8aac72886d563851b966bf18/typing_extensions-4.2.0.tar.gz#sha256=f1c24655a0da0d1b67f07e17a5e6b2a105894e6824b92096378bb3668ef02376" data-requires-python="&gt;=3.7">typing_extensions-4.2.0.tar.gz</a><br/>
<a href="https://files.pythonhosted.org/packages/4d/16/0a14ca596f30316efd412a60bdfac02a7259bf8673d4d917dc60b9a21812/pip-22.0.4-py3-none-any.whl#sha256=c6aca0f2f081363f689f041d90dab2a07a9a07fb840284db2218117a52da800b" data-requires-python="&gt;=3.7">pip-22.0.4-py3-none-any.whl</a><br/>
<a href="https://files.pythonhosted.org/packages/33/c9/e2164122d365d8f823213a53970fa3005eb16218edcfc56ca24cb6deba2b/pip-22.0.4.tar.gz#sha256=b3a9de2c6ef801e9247d1527a4b16f92f2cc141cd1489f3fffaf6a9e96729764" data-requires-python="&gt;=3.7">pip-22.0.4.tar.gz</a><br/>
uranusjr commented 2 years ago

Hmm, this works correctly on Ubuntu for me (I only have 3.6.13 but that shouldn’t affect things…?)

$ virtualenv --python=python3.6 pip-11046
created virtual environment CPython3.6.13.final.0-64 in 460ms
  creator CPython3Posix(dest=/home/uranusjr/Documents/play/pip-11046, clear=False, no_vcs_ignore=False, global=False)
  seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/home/uranusjr/.local/share/virtualenv)
    added seed packages: pip==21.3.1, setuptools==59.1.1, wheel==0.37.0
$ ./pip-11046/bin/python -m pip install -v --no-cache-dir --disable-pip-version-check typing-extensions
Using pip 21.3.1 from /home/uranusjr/Documents/play/pip-11046/lib/python3.6/site-packages/pip (python 3.6)
  Link requires a different Python (3.6.13 not in: '>=3.7'): https://files.pythonhosted.org/packages/75/e1/932e06004039dd670c9d5e1df0cd606bf46e29a28e65d5bb28e894ea29c9/typing_extensions-4.2.0-py3-none-any.whl#sha256=6657594ee297170d19f67d55c05852a874e7eb634f4f753dbd667855e07c1708 (from https://pypi.org/simple/typing-extensions/) (requires-python:>=3.7)
  Link requires a different Python (3.6.13 not in: '>=3.7'): https://files.pythonhosted.org/packages/fe/71/1df93bd59163c8084d812d166c907639646e8aac72886d563851b966bf18/typing_extensions-4.2.0.tar.gz#sha256=f1c24655a0da0d1b67f07e17a5e6b2a105894e6824b92096378bb3668ef02376 (from https://pypi.org/simple/typing-extensions/) (requires-python:>=3.7)
Collecting typing-extensions
  Downloading typing_extensions-4.1.1-py3-none-any.whl (26 kB)
Installing collected packages: typing-extensions
Successfully installed typing-extensions-4.1.1

Can you try pip install -vv --no-cache-dir --disable-pip-version-check typing-extensions? pip would print all the links it found in double-verbose mode. I wonder if pip somehow does not sees the requires-python attribute, or magically thinks 3.6.11 matches >=3.7.

jtratner commented 2 years ago

Thanks @uranusjr - I will test this out on more systems! (FWIW this started as an issue on ubuntu [but prob old Ubuntu] and then I reproduced it on my mac)

jtratner commented 2 years ago

I am SO SO sorry, I'm so embarrassed. I realized we'd put in a config file to ignore requires-python earlier and I didn't notice it until the -vv check.

Sorry to waste your time there - I should've tried to reproduce this without having a pip config file set up.

jtratner commented 2 years ago 
  Ignoring failed Requires-Python check (3.6.11 not in: '>=3.7') for link: https://files.pythonhosted.org/packages/fe/71/1df93bd59163c8084d812d166c907639646e8aac72886d563851b966bf18/typing_extensions-4.2.0.tar.gz#sha256=f1c24655a0da0d1b67f07e17a5e6b2a105894e6824b92096378bb3668ef02376 (from https://pypi.org/simple/typing-extensions/) (requires-python:>=3.7)