Unable to parse requirements file containing arguments to pip

[bendoerry]

We have pip requirements file that contains an argument to pip.

pip is able to install from this file without any issues, but running LicenseFinder against it with

license_finder --pip-requirements-path=requirements.lock --python-version=3

results in the following traceback

LicenseFinder::Pip: is active
pip3 install: did not succeed.
pip3 install: LicenseFinder command 'python3 /var/lib/gems/2.7.0/gems/license_finder-6.14.2/bin/license_finder_pip.py /.../requirements.lock' failed:
    Traceback (most recent call last):
  File "/var/lib/gems/2.7.0/gems/license_finder-6.14.2/bin/license_finder_pip.py", line 27, in <module>
    if req.req is not None and (req.markers is None or req.markers.evaluate()):
       ^^^^^^^
AttributeError: 'ParsedRequirement' object has no attribute 'req'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/pip/_vendor/packaging/requirements.py", line 102, in __init__
    req = REQUIREMENT.parseString(requirement_string)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/pip/_vendor/pyparsing/core.py", line 1141, in parse_string
    raise exc.with_traceback(None)
pip._vendor.pyparsing.exceptions.ParseException: Expected string_end, found ':'  (at char 4), (line:1, col:5)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/pip/_vendor/pkg_resources/__init__.py", line 3101, in __init__
    super(Requirement, self).__init__(requirement_string)
  File "/usr/local/lib/python3.11/site-packages/pip/_vendor/packaging/requirements.py", line 104, in __init__
    raise InvalidRequirement(
pip._vendor.packaging.requirements.InvalidRequirement: Parse error at "':.'": Expected string_end

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/var/lib/gems/2.7.0/gems/license_finder-6.14.2/bin/license_finder_pip.py", line 32, in <module>
    reqs.append(pkg_resources.Requirement.parse(str(req.requirement)))
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/pip/_vendor/pkg_resources/__init__.py", line 3148, in parse
    req, = parse_requirements(s)
    ^^^^
  File "/usr/local/lib/python3.11/site-packages/pip/_vendor/pkg_resources/__init__.py", line 3094, in parse_requirements
    yield Requirement(line)
          ^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/pip/_vendor/pkg_resources/__init__.py", line 3103, in __init__
    raise RequirementParseError(str(e))
pip._vendor.pkg_resources.RequirementParseError: Parse error at "':.'": Expected string_end

The traceback is from version 6.14.2, but I have confirmed we get the same behaviour with version 7.0.1.

Removing the pip argument from the file results in LicenseFinder running as expected.

[cf-gitbot]

We have created an issue in Pivotal Tracker to manage this. Unfortunately, the Pivotal Tracker project is private so you may be unable to view the contents of the story.

The labels on this github issue will be updated when the story is started.

[suraj2410]

@bendoerry does this work for you? i am facing this same issue with having an internal package with git+ssh

pip3 install: LicenseFinder command 'python3 /var/lib/gems/3.0.0/gems/license_finder-7.2.1/bin/license_finder_pip.py /home/runner/work/project/requirements.txt' failed:
    /var/lib/gems/3.0.0/gems/license_finder-7.2.1/bin/license_finder_pip.py:20: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
  from pip._vendor import pkg_resources
Traceback (most recent call last):
  File "/var/lib/gems/3.0.0/gems/license_finder-7.2.1/bin/license_finder_pip.py", line 27, in <module>
    if req.req is not None and (req.markers is None or req.markers.evaluate()):
AttributeError: 'ParsedRequirement' object has no attribute 'req'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/home/runner/.local/lib/python3.10/site-packages/pip/_vendor/packaging/requirements.py", line 102, in __init__
    req = REQUIREMENT.parseString(requirement_string)
  File "/home/runner/.local/lib/python3.10/site-packages/pip/_vendor/pyparsing/util.py", line 256, in _inner
    return fn(self, *args, **kwargs)
  File "/home/runner/.local/lib/python3.10/site-packages/pip/_vendor/pyparsing/core.py", line 1190, in parse_string
    raise exc.with_traceback(None)
pip._vendor.pyparsing.exceptions.ParseException: Expected string_end, found '+'  (at char 3), (line:1, col:4)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/var/lib/gems/3.0.0/gems/license_finder-7.2.1/bin/license_finder_pip.py", line 32, in <module>
    reqs.append(pkg_resources.Requirement.parse(str(req.requirement)))
  File "/home/runner/.local/lib/python3.10/site-packages/pip/_vendor/pkg_resources/__init__.py", line 3215, in parse
    (req,) = parse_requirements(s)
  File "/home/runner/.local/lib/python3.10/site-packages/pip/_vendor/pkg_resources/__init__.py", line 3174, in __init__
    super(Requirement, self).__init__(requirement_string)
  File "/home/runner/.local/lib/python3.10/site-packages/pip/_vendor/packaging/requirements.py", line 104, in __init__
    raise InvalidRequirement(
pip._vendor.packaging.requirements.InvalidRequirement: Parse error at "'+ssh://g'": Expected string_end

Also tried to use pip version 20.0.2 but same error above