Install failed for pants plugin

[kaos]

Describe the bug Well this was annoying.

ERROR: Cannot install acme-pants-plugin==2.15.0.dev8+2.21.0 because these package versions have conflicting dependencies.
ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/topics/dependency-resolution/#dealing-with-dependency-conflicts

 The conflict is caused by:
     acme-pants-plugin 2.15.0.dev8+2.21.0 depends on pantsbuild.pants==2.21.0
     The user requested (constraint) pantsbuild-pants==2.21.0

 To fix this you could try to:
 1. loosen the range of package versions you've specified
 2. remove package versions to allow pip attempt to solve the dependency conflict

Pants version 2.21.0

OS Mac.

Additional info Took me a second to spot the difference.. (. vs -.)

[GLOBAL]
pants_version = "2.21.0"
plugins = ["acme-pants-plugin==2.15.0.dev8+2.21.0"]

❯ cat acme_pants_plugin-0.0.0.dist-info/METADATA
Metadata-Version: 2.1
Name: acme-pants-plugin
Version: 0.0.0
Requires-Python: ==3.9.*
Requires-Dist: click
Requires-Dist: pantsbuild.pants (==2.21.0)

The pants_requirements() BUILD file target results in a dependency on pantsbuild.pants:

❯ pants peek 3rdparty:pantsbuild
[
  {
    "address": "3rdparty:pantsbuild",
    "target_type": "pants_requirements",
    "_find_links": [],
    "dependencies": [
      "3rdparty:pantsbuild#pantsbuild.pants",
      "3rdparty:pantsbuild#pantsbuild.pants.testutil"
    ],
    "description": null,
    "tags": null,
    "testutil": true,
    "version_spec": "== 2.21.0"
  }
]

The constraints passed to PEX/pip also lists pantsbuild.pants, so this seems it could be a dist name normalization applied by PEX/pip. This normalization seems to contradict https://peps.python.org/pep-0503/#normalized-names

Edit: The names and versions are ~irrelevant. What matters is that the pantsbuild distro is not available on PyPI since 2.18.0, and scie-pants doesn't provide the find-links option to locate them in the custom wheel shop for plugins to consume.

[kaos]

This comes from: https://github.com/pypa/pip/blob/71b168879e75796b61f524bfcedc99b1de72f58b/src/pip/_vendor/packaging/utils.py#L45-L50

_canonicalize_regex = re.compile(r"[-_.]+")

def canonicalize_name(name: str, *, validate: bool = False) -> NormalizedName:
    if validate and not _validate_regex.match(name):
        raise InvalidName(f"name is invalid: {name!r}")
    # This is taken from PEP 503.
    value = _canonicalize_regex.sub("-", name).lower()
    return cast(NormalizedName, value)

Although, the motivations seems to point at PEP-503, it doesn't not rule out the use of . in package names, however, this has been done like this for almost a decade, so seems more likely to be easier to adapt to this world rather than fix this upstream.

[kaos]

This may have been solved by https://github.com/pypa/pip/commit/767bb40755243e1519be262c44abbda81d06868b but we should still produce normalized package names according to what PyPA expects.

[kaos]

Hmm.. using an older Pants version (and thus older pex/pip) I think I get a more accurate error message (in this case, I think it is the same root cause):

ERROR: Could not find a version that satisfies the requirement pantsbuild.pants==2.21.0 (from acme-pants-plugin)
ERROR: No matching distribution found for pantsbuild.pants==2.21.0

But, I wonder, it seems like pantsbuild.pants ought to exist in the venv that we try to resolve the plugin loading in, but it doesn't seem to be there.

[kaos]

OK- took me way longer than I care to admit, but the root cause is the absence of pantsbuild.pants on PyPI. When I use pants version 2.18.0a0 it works, but for 2.18.0 it fails, on the exact same setup. The only difference being that 2.18.0a0 is the last version available on PyPI.

I'm surprised no one else have run into this yet..

The work-around is to add this to your pants.toml:

[python-repos]
find_links = [
  "https://wheels.pantsbuild.org/simple",
]

I've ruled out scie-pants as the source of this regression, by confirming this work-around is required also for scie-pants 0.10.0, which is the oldest version supporting pants >=2.18.0.dev6.