search

aboutcode-org / python-inspector

Inspect Python code and PyPI package manifests. Resolve Python dependencies.
22 stars 19 forks source link

Parse error at "'# Note: '": Expected W:(0-9A-Za-z) #152

Open bennati opened 1 year ago

bennati commented 1 year ago

python inspector 0.9.8 fails to parse a requirements.txt file containing:

molecule==2.22
taskcat==0.9.20

This error does not happen with python inspector 0.9.4

Stacktrace:

> python-inspector --python-version 310 --operating-system linux --json-pdt o.json --analyze-setup-py-insecurely --requirement requirements.txt
Traceback (most recent call last):
  File "/Users/bennati/.pyenv/versions/scancode/lib/python3.8/site-packages/packvers/requirements.py", line 102, in __init__
    req = REQUIREMENT.parseString(requirement_string)
  File "/Users/bennati/.pyenv/versions/scancode/lib/python3.8/site-packages/pyparsing/core.py", line 1141, in parse_string
    raise exc.with_traceback(None)
pyparsing.exceptions.ParseException: Expected W:(0-9A-Za-z), found '#'  (at char 0), (line:1, col:1)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/bennati/.pyenv/versions/scancode/lib/python3.8/site-packages/python_inspector/resolve_cli.py", line 247, in resolve_dependencies
    resolution_result: Dict = resolver_api(
  File "/Users/bennati/.pyenv/versions/scancode/lib/python3.8/site-packages/python_inspector/api.py", line 263, in resolve_dependencies
    resolution, purls = resolve(
  File "/Users/bennati/.pyenv/versions/scancode/lib/python3.8/site-packages/python_inspector/api.py", line 322, in resolve
    resolved_dependencies, packages = get_resolved_dependencies(
  File "/Users/bennati/.pyenv/versions/scancode/lib/python3.8/site-packages/python_inspector/api.py", line 360, in get_resolved_dependencies
    resolver_results = resolver.resolve(requirements=requirements, max_rounds=max_rounds)
  File "/Users/bennati/.pyenv/versions/scancode/lib/python3.8/site-packages/resolvelib/resolvers.py", line 546, in resolve
    state = resolution.resolve(requirements, max_rounds=max_rounds)
  File "/Users/bennati/.pyenv/versions/scancode/lib/python3.8/site-packages/resolvelib/resolvers.py", line 427, in resolve
    failure_causes = self._attempt_to_pin_criterion(name)
  File "/Users/bennati/.pyenv/versions/scancode/lib/python3.8/site-packages/resolvelib/resolvers.py", line 239, in _attempt_to_pin_criterion
    criteria = self._get_updated_criteria(candidate)
  File "/Users/bennati/.pyenv/versions/scancode/lib/python3.8/site-packages/resolvelib/resolvers.py", line 229, in _get_updated_criteria
    for requirement in self._p.get_dependencies(candidate=candidate):
  File "/Users/bennati/.pyenv/versions/scancode/lib/python3.8/site-packages/python_inspector/resolution.py", line 644, in get_dependencies
    return list(self._iter_dependencies(candidate))
  File "/Users/bennati/.pyenv/versions/scancode/lib/python3.8/site-packages/python_inspector/resolution.py", line 635, in _iter_dependencies
    for r in self.get_requirements_for_package(purl=purl, candidate=candidate):
  File "/Users/bennati/.pyenv/versions/scancode/lib/python3.8/site-packages/python_inspector/resolution.py", line 497, in get_requirements_for_package_from_pypi_simple
    yield from get_reqs_insecurely(
  File "/Users/bennati/.pyenv/versions/scancode/lib/python3.8/site-packages/python_inspector/resolution.py", line 277, in get_reqs_insecurely
    yield from parse_reqs_from_setup_py_insecurely(setup_py=setup_py_location)
  File "/Users/bennati/.pyenv/versions/scancode/lib/python3.8/site-packages/python_inspector/resolution.py", line 128, in parse_reqs_from_setup_py_insecurely
    for req in iter_requirements(level="", extras=[], setup_file=setup_py):
  File "/Users/bennati/.pyenv/versions/scancode/lib/python3.8/site-packages/python_inspector/setup_py_live_eval.py", line 151, in iter_requirements
    pkg = Requirement(req)
  File "/Users/bennati/.pyenv/versions/scancode/lib/python3.8/site-packages/packvers/requirements.py", line 104, in __init__
    raise InvalidRequirement(
packvers.requirements.InvalidRequirement: Parse error at "'# Note: '": Expected W:(0-9A-Za-z)
35C4n0r commented 11 months ago

@TG1999 I would like to take up this issue.

TG1999 commented 11 months ago

@35C4n0r go ahead!

35C4n0r commented 11 months ago

@TG1999 To handle/ignore comments properly in the requirenments.txt files, we would have to handle it in the packvers. I'll go ahead and make a PR there.

35C4n0r commented 11 months ago

@TG1999 these are the PRs, Kindly review them. https://github.com/nexB/packvers/pull/3, https://github.com/nexB/python-inspector/pull/166