jazzband / pip-tools

A set of tools to keep your pinned Python dependencies fresh.
https://pip-tools.rtfd.io
BSD 3-Clause "New" or "Revised" License
7.75k stars 610 forks source link

WIP: Handle broken metadata by discarding package, or give a useful error #2119

Open khink opened 3 months ago

khink commented 3 months ago

Fixes #2118, but no tests yet (hence WIP).

After this change, a requirements file is generated by discarding the problematic package.

Now, with this in requirements.txt:

django-debug-toolbar
django-geojson
wagtail>=4.2,<4.3

I get:

$ env/bin/pip-compile --resolver=backtracking --generate-hashes --allow-unsafe --output-file requirements.txt requirements.in
    error: subprocess-exited-with-error

    × python setup.py egg_info did not run successfully.
    │ exit code: 1
    ╰─> [18 lines of output]
        /home/kees/Projects/pip-tools/env/lib/python3.10/site-packages/setuptools/_distutils/dist.py:268: UserWarning: Unknown distribution option: 'test_suite'
          warnings.warn(msg)
        Traceback (most recent call last):
          File "<string>", line 2, in <module>
          File "<pip-setuptools-caller>", line 34, in <module>
          File "/tmp/pip-resolve-o2r8fhgd/geojson_90f4efbe8a2247f9a1883c03e4988381/setup.py", line 6, in <module>
            setup(name          = "geojson",
          File "/home/kees/Projects/pip-tools/env/lib/python3.10/site-packages/setuptools/__init__.py", line 108, in setup
            return distutils.core.setup(**attrs)
          File "/home/kees/Projects/pip-tools/env/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 146, in setup
            _setup_distribution = dist = klass(attrs)
          File "/home/kees/Projects/pip-tools/env/lib/python3.10/site-packages/setuptools/dist.py", line 289, in __init__
            self.metadata.version = self._normalize_version(self.metadata.version)
          File "/home/kees/Projects/pip-tools/env/lib/python3.10/site-packages/setuptools/dist.py", line 325, in _normalize_version
            normalized = str(Version(version))
          File "/home/kees/Projects/pip-tools/env/lib/python3.10/site-packages/packaging/version.py", line 200, in __init__
            match = self._regex.search(version)
        TypeError: cannot use a string pattern on a bytes-like object
        [end of output]

    note: This error originates from a subprocess, and is likely not a problem with pip.
Discarding django-geojson (from -r requirements.in (line 11)) to proceed the resolution.
It looks like it has a release with broken metadata, causing this resolver to fail.
You may try to pin that package's version (probably to a recent one), so the resolver doesn't consider the broken release.

The exception is still shown. Also we log a warning to tell the user the package was discarded.

The way to find the offending pacakge is crude (regex in error message string).

When the regex search for package name fails, we re-raise the error so no requirements file is generated.

In that case we log an error, which is hopefully helpful.

Contributor checklist
Maintainer checklist