PyPI's JSON API response for project handler deprecates releases information

[pilosus]

Recently PyPI's JSON API deprecated releases key in project's response:

The releases key on this response should be considered deprecated, and projects should shift to using the simple API (which can be accessed as JSON via PEP 691) to get this information where possible.

In the future, the releases key may be removed from this response.

That breaks versions resolution built in the checker for Python deps.

There are two ways to fix that:

1. Switch over JSON based Simple API for packge indicies for version resolution. E.g.

Instead of getting all available releases from JSON API's releases section:

curl -SsL 'https://pypi.org/pypi/<project>/json' | jq '.releases|keys' | sort

Use:

 curl -SsL --header 'Accept: application/vnd.pypi.simple.v1+json' --url 'https://pypi.org/simple/<project>' | jq

get all available files, parse versions from files, calculate version, then obtain meta from JSON API.

Pro: will fix the issue Contra:

• Filename parsing implementation needed
• JSON-based simple API response PEP 691 from private indicies may be still be not supported, so HTML response PEP 503 shall be also supported
• All non-PyPI and PyPI-like private indices (VCS, local project directories or remote source archives) checks still cannot be supported (see #107)

2. Give up on getting Python package's license name from API completely (i.e. drop --requirements option), support only license name to license category functionality (i.e. take in license names as with --external option).

Pro:

• No need to support Python-specific things (e.g. version resolution)
• Code simplification
• Easier to focus on one thing and try to do it well

Contra:

• Drastic project reorganisation with backward compatibility breaks (how come it's called pip-something if it has nothing to do with Python directly?)

[pilosus]

There's a proposal to user versions attribute for JSON API: https://github.com/pypi/warehouse/issues/11991 https://github.com/pypi/warehouse/pull/12079

But it's still not released. There's also a concern that the project endpoint in JSON API may return too much (e.g. project's long description), which is bad for performance. So the Simple API is preferrable.

PEP 658 also defines an attribute for metadata for Simple API. PEP 691 defines an optional key dist-info-metadata in JSON response for Simple API to indicate if a separate metadata file is available. This way we could switch over JSON-based Simple API completely.

[pilosus]

When parsing versions from Simple API, we must rely on PEP-427 that defines filename naming conventions.

[pilosus]

The most up-to-date information on Python distribution files can be found under: https://packaging.python.org/en/latest/specifications/

See Package Distribution File Formats for sdist and wheel files specs