Direct reference metadata compatibility

[ofek]

I'm in the process of adding support for UV in Hatch and I noticed that the test suite is failing because of a difference in how UV writes direct_url.json vs pip, specifically the requested_revision field. The reason this is read is to determine whether dependencies are in sync.

Given the following requirements:

requests @ git+https://github.com/psf/requests
httpx @ git+https://github.com/encode/httpx@master

The following command shows different results based on which installer was used:

python -c "import json,pprint;from importlib.metadata import distribution;pprint.pprint([json.loads(distribution(pkg).read_text('direct_url.json')) for pkg in ['requests', 'httpx']])"

UV:

[{'url': 'https://github.com/psf/requests',
  'vcs_info': {'commit_id': '31ebb8102c00f8cf8b396a6356743cca4362e07b',
               'requested_revision': '31ebb8102c00f8cf8b396a6356743cca4362e07b',
               'vcs': 'git'}},
 {'url': 'https://github.com/encode/httpx',
  'vcs_info': {'commit_id': '4b85e6c3898b94e686b427afd83138c87520b479',
               'requested_revision': '4b85e6c3898b94e686b427afd83138c87520b479',
               'vcs': 'git'}}]

pip:

[{'url': 'https://github.com/psf/requests',
  'vcs_info': {'commit_id': '31ebb8102c00f8cf8b396a6356743cca4362e07b',
               'vcs': 'git'}},
 {'url': 'https://github.com/encode/httpx',
  'vcs_info': {'commit_id': '4b85e6c3898b94e686b427afd83138c87520b479',
               'requested_revision': 'master',
               'vcs': 'git'}}]

[charliermarsh]

Thanks, I can take a look.

[pradyunsg]

FWIW, I think the spec doesn't make it clear that the expectation is for the mismatched field (requested revision) to match what was originally specified, rather than the final commit hash.

This clarification is worth a spec update PR on the packaging.python.org page.

[ofek]

I think the requested revision should match what the user specified (or did not) indeed. If you want to encode more data it looks like the PEP has this text which was not ported over to the living document spec:

• If the installer could discover additional information about the requested revision, it MAY add a resolved_revision and/or resolved_revision_type field. If no revision was provided in the requested URL, resolved_revision MAY contain the default branch that was installed, and resolved_revision_type will be branch. If the installer determines that requested_revision was a tag, it MAY add resolved_revision_type with value tag.

[ofek]

This clarification is worth a spec update PR on the packaging.python.org page.

https://github.com/pypa/packaging.python.org/pull/1531

[ofek]

The text has been updated: https://packaging.python.org/en/latest/specifications/direct-url-data-structure/#vcs-urls

[charliermarsh]

Thanks. Probably not gonna fix it immediately, will require some internal refactors.

[ofek]

Have the refactors been completed?