2.2.0 will install only on Python 3.6, not on 3.7 or 3.8

[nickwilliams-eventbrite]

Due to this line in setup.cfg:

https://github.com/jaraco/zipp/blob/v2.2.0/setup.cfg#L17

The wheel published to PyPi is limited to Python 3.6 and will not install on Python 3.7 or 3.8 (name has py36 and nothing else in it: zipp-2.2.0-py36-none-any.whl):

https://files.pythonhosted.org/packages/46/42/f2dd964b2a6b1921b08d661138148c1bcd3f038462a44019416f2342b618/zipp-2.2.0-py36-none-any.whl

The [options] say python_version >= 3.6, but the binary distribution setting is 3.6-only. One of these is not correct, and I suspect it's binary distribution setting. Assuming it is, the python-tag needs to be changed to either python-tag=py36.py37.py38 or to universal=1.

[jaraco]

I was worried about this. In #37, @vfazio made the case that py36 means "python 3.6 or later".

And in my testing, that's what I see.

zipp master $ python3.8 -m pip-run --no-cache-dir zipp -- -c 'pass'
Collecting zipp
  Downloading zipp-2.2.0-py36-none-any.whl (4.6 kB)
Installing collected packages: zipp
Successfully installed zipp-2.2.0

Can you describe where and how this package installs on Python 3.6 only?

[vfazio]

Is this a theoretical problem or an actual problem?

[vfazio]

root@ce09ab0be78c:/# mktemp -d
/tmp/tmp.NOhEkFwhkQ
root@ce09ab0be78c:/# cd /tmp/tmp.NOhEkFwhkQ/
root@ce09ab0be78c:/tmp/tmp.NOhEkFwhkQ# python3 -m venv venv
root@ce09ab0be78c:/tmp/tmp.NOhEkFwhkQ# . venv/bin/activate
(venv) root@ce09ab0be78c:/tmp/tmp.NOhEkFwhkQ# python -V
Python 3.7.5
(venv) root@ce09ab0be78c:/tmp/tmp.NOhEkFwhkQ# pip search zipp 
zipp (2.2.0)  - Backport of pathlib-compatible object wrapper for zip files
(venv) root@ce09ab0be78c:/tmp/tmp.NOhEkFwhkQ# pip install wheel
Collecting wheel
  Downloading https://files.pythonhosted.org/packages/8c/23/848298cccf8e40f5bbb59009b32848a4c38f4e7f3364297ab3c3e2e2cd14/wheel-0.34.2-py2.py3-none-any.whl
Installing collected packages: wheel
Successfully installed wheel-0.34.2
(venv) root@ce09ab0be78c:/tmp/tmp.NOhEkFwhkQ# pip install zipp
Collecting zipp
  Downloading https://files.pythonhosted.org/packages/46/42/f2dd964b2a6b1921b08d661138148c1bcd3f038462a44019416f2342b618/zipp-2.2.0-py36-none-any.whl
Installing collected packages: zipp
Successfully installed zipp-2.2.0
(venv) root@ce09ab0be78c:/tmp/tmp.NOhEkFwhkQ# 

[nickwilliams-eventbrite]

On our Python 3.7.6 system running on Ubuntu 16.04 and pip 19.3.1, we get the error "Could not find a version that satisfies the requirement zipp==2.2.0." It will let us install 2.1.0 and below, which does not have that line in setup.cfg, but not 2.2.0, in which that line was added. In our experience, when the version exists in PyPi but pip still spits out that error, it actually means that it couldn't find a matching version compatible with your Python version.

We manage dozens of libraries, both open source and internally, and we've always had to do something like python-tag=version.version.version to support installing on multiple versions, kind of like this: https://github.com/eventbrite/pysoa/blob/1d6fa4bd/setup.cfg#L2. My recommendation for this library is to change python-tag=py36 to python-tag=py36.py37.py38. That will create a wheel named zipp-2.2.1-py36.py37.py38-none-any.whl.

[vfazio]

The ubuntu distribution does some weird things with versioning... just an FYI

[nickwilliams-eventbrite]

The ubuntu distribution does some weird things with versioning... just an FYI

We're actually not using the Ubuntu distribution. We're downloading, compiling, and installing from python.org, because the Ubuntu distribution of Python 3.7 is hopelessly broken.

[vfazio]

FYI

(venv) root@ce09ab0be78c:/tmp/tmp.NOhEkFwhkQ# pip install pip==19.3.1
Collecting pip==19.3.1
  Downloading https://files.pythonhosted.org/packages/00/b6/9cfa56b4081ad13874b0c6f96af8ce16cfbc1cb06bedf8e9164ce5551ec1/pip-19.3.1-py2.py3-none-any.whl (1.4MB)
    100% |████████████████████████████████| 1.4MB 768kB/s 
Installing collected packages: pip
  Found existing installation: pip 18.1
    Uninstalling pip-18.1:
      Successfully uninstalled pip-18.1
Successfully installed pip-19.3.1
(venv) root@ce09ab0be78c:/tmp/tmp.NOhEkFwhkQ# pip uninstall zipp
Uninstalling zipp-2.2.0:
  Would remove:
    /tmp/tmp.NOhEkFwhkQ/venv/lib/python3.7/site-packages/zipp-2.2.0.dist-info/*
    /tmp/tmp.NOhEkFwhkQ/venv/lib/python3.7/site-packages/zipp.py
Proceed (y/n)? y
  Successfully uninstalled zipp-2.2.0
(venv) root@ce09ab0be78c:/tmp/tmp.NOhEkFwhkQ# pip install zipp
Collecting zipp
  Using cached https://files.pythonhosted.org/packages/46/42/f2dd964b2a6b1921b08d661138148c1bcd3f038462a44019416f2342b618/zipp-2.2.0-py36-none-any.whl
Installing collected packages: zipp
Successfully installed zipp-2.2.0
WARNING: You are using pip version 19.3.1; however, version 20.0.2 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
(venv) root@ce09ab0be78c:/tmp/tmp.NOhEkFwhkQ# cat /etc/os-release 
NAME="Ubuntu"
VERSION="19.10 (Eoan Ermine)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 19.10"
VERSION_ID="19.10"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=eoan
UBUNTU_CODENAME=eoan

[vfazio]

what version of setuptools is installed?

[jaraco]

My guess is that @nickwilliams-eventbrite is not installing from PyPI, but is installing from some other index or from a file system. @nickwilliams-eventbrite Do you have a way that we might be able to replicate your finding?

[nickwilliams-eventbrite]

Ah, @jaraco does make a good point. We're installing from an internal Devpi 5.3.0 server, not directly from PyPi. This Devpi server mirrors PyPi, and 2.2.0 is listed on the Devpi server simple index, and we can install 2.2.0 on Python 3.6 from the Devpi server, just not on Python 3.7.

[vfazio]

i think that's a limitation of the server software then? we host using Nexus as a mirror for upstream pypi and we haven't had trouble (yet)

[nickwilliams-eventbrite]

Perhaps this is a limitation (they may argue a "feature") of Devpi, but there's also a super simple fix that we already know will work (because it already works for us), which is to change the Python tag to python-tag=py36.py37.py38, which will create a wheel named zipp-2.2.1-py36.py37.py38-none-any.whl that we would be able to install. Another option that we know will work is to remove python-tag entirely. It doesn't appear to be needed to create a wheel, as Django publishes a wheel but does not have python-tag or universal: https://github.com/django/django/blob/master/setup.cfg

[vfazio]

the challenge i've had with the latter is that python 3.5 will fetch it but can't install it

[nickwilliams-eventbrite]

In my experience, python_requires = >=3.6 (which you already have) prevents that. We get the "Could not find a version" error for 2.1.0 on Python 3.5.

[vfazio]

easy way to test this, give me a sec

[nickwilliams-eventbrite]

If you look at the PyPi simple index for Zipp: https://pypi.org/simple/zipp/

And right-click on a link for 2.1.0 and click "Inspect" from the context menu, you'll see that the link contains the attribute data-requires-python=">=3.6". This is what prevents Pip from attempting to install on anything less than 3.6.

[jaraco]

It's not a big surprise to me that other tools like DevPi may have had a different interpretation of the Python tags than vfazio. I've reached out in pypa/packaging-problems#322 to get clarity on what is the intention for these tags. Once we have clarity on that, we can address that across the ecosystem. Until then, I think the right thing to do is to go back to py3 as the tag.

change the Python tag to python-tag=py36.py37.py38

I don't believe the solution is as simple as you say. First, that tag may be correct today for most production environments, but it will be incorrect once Python 3.9 comes out, plus it creates a lot of flux keeping each supported minor version declared in the package names.

Django publishes a wheel but does not have python-tag or universal.

If you don't have python-tag or universal, then the wheels will be released with whatever Python version is being used to cut the release. And it looks like the tag that's used is py3 (https://pypi.org/project/Django/#files), which is what zipp had prior to #37.

[vfazio]

apparently data-requires-python is an optional feature of PEP 503 :-/

[nickwilliams-eventbrite]

apparently data-requires-python is an optional feature of PEP 503 :-/

Oh that's unfortunate. Thankfully, though, both PyPi and Devpi implement it. Not sure about Nexus, but PyPi and Devpi together cover a vast majority of the ecosystem.

[jaraco]

vfazio gave a demo in this comment where it seems that pip relies on the tags when installing from dists in the file system.

[vfazio]

I double checked my cfg file for those tests. i only changed python-tag:

[bdist_wheel]
python-tag=py35

[options]
py_modules = zipp
packages = find:
include_package_data = true
python_requires = >=3.6

[jaraco]

Not sure about Nexus

I'm pretty sure I've seen that Nexus supports it also. Tracker indicates support was added in 3.20.0.

[vfazio]

Not sure about Nexus

I'm pretty sure I've seen that Nexus supports it also. Tracker indicates support was added in 3.20.0.

Correct, i just tested this against our instance of 3.20.1 (though we were on an older version prior which was maybe why we were getting errors in our tox tests because incompatible versions were being fetched)

[vfazio]

note also that the django config you pointed to isn't actually released yet AFAICT.

Anyway, i see the following:

• Devpi should support python-tag values
• get more info from the packaging maintainers
• slap py36.py37.py38 on it, though i really think py36 is sufficient based on my reading of the PEP

[pfmoore]

I haven't been able to read all of this discussion, but Python 3.7 should be able to install a .py36 wheel. pip debug --verbose will show you the compatible tags. On my (Windows Python 3.8) system, I get

Compatible tags: 30
  cp38-cp38-win_amd64
  cp38-abi3-win_amd64
  cp38-none-win_amd64
  cp37-abi3-win_amd64
  cp36-abi3-win_amd64
  cp35-abi3-win_amd64
  cp34-abi3-win_amd64
  cp33-abi3-win_amd64
  cp32-abi3-win_amd64
  py38-none-win_amd64
  py3-none-win_amd64
  py37-none-win_amd64
  py36-none-win_amd64
  py35-none-win_amd64
  py34-none-win_amd64
  py33-none-win_amd64
  py32-none-win_amd64
  py31-none-win_amd64
  py30-none-win_amd64
  cp38-none-any
  py38-none-any
  py3-none-any
  py37-none-any
  py36-none-any
  py35-none-any
  py34-none-any
  py33-none-any
  py32-none-any
  py31-none-any
  py30-none-any

Note that py36-none-any is in there. I should point out that there has been a reasonable amount of change in the actual tag code recently, so if you're using an older version of pip, or a debundled version of packaging, or anything like that, you should check your system.

[jaraco]

* Devpi should support python-tag values

I think there's consensus here that Devpi should interpret python-tag values as greater-than-or-equal. @nickwilliams-eventbrite Would you be willing to file that with that project? You may want to consider getting a commercial support contract through Merlinux don't already.

* slap py36.py37.py38 on it, though i really think py36 is sufficient based on my reading of the PEP

As I illustrated before, this workaround is tedious and error-prone. I'm really reluctant to add it, especially as adding it here implies it should be added across the 140 projects I maintain (and thousands of others I don't).

I'm more inclined to roll back to simply py3 as the indicated version or no tag at all (as Django has done since it dropped support for Python 2) and let the tool chains work out the best version to advertise. The solution simply can't be to push a bunch of toil on the packagers.

[vfazio]

I think the difference here is that django is supported on all py3. it doesn't have a min version requirement. This is changing on master however, but it has not been cut and released with this change AFACT

[nickwilliams-eventbrite]

I think the difference here is that django is supported on all py3. it doesn't have a min version requirement. This is changing on master however, but it has not been cut and released with this change AFACT

Django 3.0 has been released to PyPi and is restricted to Python >= 3.6 (it is not supported on all Py3) with that setup.cfg without a python-tag: https://pypi.org/project/Django/3.0.3/

[vfazio]

yea i see it now.. it's just a little obtuse https://github.com/django/django/blob/3.0.3/setup.py#L72

[jaraco]

For posterity, here's a Dockerfile I was using to replicate the issue. I didn't get around to installing from devpi.

FROM ubuntu:xenial

RUN apt update
RUN apt install -y software-properties-common
RUN apt-add-repository ppa:deadsnakes/ppa
RUN apt update
RUN apt install -y python3.7-dev wget
RUN wget https://bootstrap.pypa.io/get-pip.py -O - | python3.7
RUN python3.7 -m pip -V

CMD python3.7 -m pip install zipp

[jaraco]

Version 2.2.1 addresses this issue using the py3 tag (the default).