search

pypi / legacy

This PyPI is no more! See https://github.com/pypa/warehouse.
Other
62 stars 46 forks source link

Recent change in package URL results order #790

Closed edmorley closed 6 years ago

edmorley commented 6 years ago

Hi!

We make use of pip's --require-hashes mode and so explicitly list package hashes in our requirements file: https://github.com/mozilla/treeherder/blob/e7743cb491d1ea20fbdc4abcb4dda1f4cb4f229a/requirements/common.txt

However after purging a local development environment virtualenv, I'm now getting hash mismatch errors for two packages:

THESE PACKAGES DO NOT MATCH THE HASHES FROM THE REQUIREMENTS FILE. If you have updated the package versions, please update the hashes. Otherwise, examine the package contents carefully; someone may have tampered with them.
    functools32==3.2.3-2 from https://pypi.python.org/packages/c5/60/6ac26ad05857c601308d8fb9e87fa36d0ebf889423f47c3502ef034365db/functools32-3.2.3-2.tar.gz#md5=09f24ffd9af9f6cd0f63cb9f4e23d4b2 (from -r requirements/common.txt (line 121)):
        Expected sha256 89d824aa6c358c421a234d7f9ee0bd75933a67c29588ce50aaa3acdf4d403fa0
             Got        f6253dfbe0538ad2e387bd8fdfd9293c925d63553f5813c4e587745416501e6d

    uritemplate==3.0.0 from https://pypi.python.org/packages/f6/25/66a49231b44409d7f07cfcf2506a8b070ce3c99fc47cc256bea833f24791/uritemplate-3.0.0-py2-none-any.whl#md5=b420ae8183a8af42481b1c655f528dfa (from -r requirements/common.txt (line 139)):
        Expected sha256 1b9c467a940ce9fb9f50df819e8ddd14696f89b9a8cc87ac77952ba416e0a8fd
             Got        01c69f4fe8ed503b2951bef85d996a9d22434d2431584b5b107b2981ff416fbd

Neither of these packages have had new uploads recently and we've pinned pip to version 9.0.1 so are using the same version as previously.

Checking the "expected" vs "got" hashes against the available downloads for each package, it seems that the hash mismatches are due to a different file being chosen rather than anything malicious. For example:

I'm presuming this is due to the recent changes to the https://pypi.python.org/simple/ API here: https://github.com/pypa/pypi-legacy/commit/66d8673d2e445499d8e7e5f491ba4703d5609e04 https://github.com/pypa/pypi-legacy/commit/85dd4477cc1c7afe8a4d71b9422d63ded447daa2 https://github.com/pypa/pypi-legacy/commit/c01a83406b118b313eb1ab0119a0cdd9af8901eb

There doesn't appear to be a PR for them so hard to see whether this behaviour change was intended?

As for our project, it's easy for me to add the additional package hashes to fix the breakage - so if this is wontfix that's fine - I just thought I should report it in case not expected. (We now use https://pyup.io to update dependencies, so future package updates will always have all hashes listed; these two packages just haven't had new versions in a while, so had not been touched by the bot.)

di commented 6 years ago

Thanks for the report @edmorley. This is expected, so I'm going to close this.

It's worth noting here that it's no longer possible for releases to have two different types of source distributions, so this really will only affect folks pinning hashes for older releases (as you have noted).

For anyone else encountering this issue, generally I'd recommend adding hashes for all distributions for a given release, possibly with a tool like hashin:

$ hashin -r requirements.txt some_project==1.2.3