search

nexB / python-inspector

Inspect Python code and PyPI package manifests. Resolve Python dependencies.
20 stars 17 forks source link

Graceful handling of non-existent packages #141

Closed tsteenbe closed 8 months ago

tsteenbe commented 11 months ago

Running Python inspector on docassemble as one of its dependencies guess-language-spirit is misspelled it's guess_language-spirit per https://pypi.org/project/guess_language-spirit/

Propose Python inspector more gracefully handled non-existent packages or packages for which it can not resolve version by still producing output and throw and error versus trowing of an error and no results.

Steps to reproduce below

pip3 install python-inspector 
python-inspector --version
Python-inspector version: 0.9.8
git clone https://github.com/jhpyle/docassemble.git
cd docassemble 
git checkout v1.4.68
cd ..
python-inspector --python-version 310 --operating-system linux --json-pdt output.json --analyze-setup-py-insecurely --setup-py docassemble/docassemble_demo/setup.py --verbose
Resolving dependencies...
direct_dependencies:
 DependentPackage(purl='pkg:pypi/docassemble', extracted_requirement='docassemble==1.4.68', scope='install')
 DependentPackage(purl='pkg:pypi/docassemble.base', extracted_requirement='docassemble.base==1.4.68', scope='install')
 DependentPackage(purl='pkg:pypi/googledrivedownloader', extracted_requirement='googledrivedownloader==0.4', scope='install')
 DependentPackage(purl='pkg:pypi/ics', extracted_requirement='ics==0.7.2', scope='install')
 DependentPackage(purl='pkg:pypi/matplotlib', extracted_requirement='matplotlib==3.7.2', scope='install')
environment: Environment(python_version='310', operating_system='linux')
repos:
 PypiSimpleRepository(index_url='https://pypi.org/simple', credentials=None)
Traceback (most recent call last):
  File "/opt/homebrew/lib/python3.11/site-packages/python_inspector/resolve_cli.py", line 247, in resolve_dependencies
    resolution_result: Dict = resolver_api(
                              ^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/python_inspector/api.py", line 263, in resolve_dependencies
    resolution, purls = resolve(
                        ^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/python_inspector/api.py", line 322, in resolve
    resolved_dependencies, packages = get_resolved_dependencies(
                                      ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/python_inspector/api.py", line 360, in get_resolved_dependencies
    resolver_results = resolver.resolve(requirements=requirements, max_rounds=max_rounds)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/resolvelib/resolvers.py", line 546, in resolve
    state = resolution.resolve(requirements, max_rounds=max_rounds)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/resolvelib/resolvers.py", line 427, in resolve
    failure_causes = self._attempt_to_pin_criterion(name)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/resolvelib/resolvers.py", line 239, in _attempt_to_pin_criterion
    criteria = self._get_updated_criteria(candidate)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/resolvelib/resolvers.py", line 230, in _get_updated_criteria
    self._add_to_criteria(criteria, requirement, parent=candidate)
  File "/opt/homebrew/lib/python3.11/site-packages/resolvelib/resolvers.py", line 148, in _add_to_criteria
    matches = self._p.find_matches(
              ^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/python_inspector/resolution.py", line 604, in find_matches
    candidates = sorted(
                 ^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/python_inspector/resolution.py", line 586, in _iter_matches
    raise NoVersionsFound(f"This package does not exist: {name}")
python_inspector.error.NoVersionsFound: This package does not exist: guess-language-spirit
tsteenbe commented 11 months ago

Fixed up the package https://github.com/tsteenbe/docassemble e.g. renaming guess-language-spirit to guess_language-spirit but Python inspector still fails with the same error message.

pombredanne commented 11 months ago

The pypi standard is to normalize the dash/underscore, so you likely found a bug.

pombredanne commented 11 months ago

The code we use for this is at https://github.com/nexB/python-inspector/blob/1d98803d7bf65196aadc0eb1490d1763b6d1af6a/src/python_inspector/utils_pypi.py#L398C13-L398C13 and we may not call it when needed.

qequ commented 11 months ago

This issue is quite similar to https://github.com/nexB/python-inspector/issues/128 and https://github.com/nexB/python-inspector/issues/127. Is it possible to add a flag to continue processing like @tsteenbe said?