Python-inspector cannot parse requirements.txt with '\' as end of line character

[gmathiou4]

When a requirements.txt file has been generated with all packages pinned to an exact version and set of hashes, like below:

numpy==1.22.1 \
    --hash=sha256:3d62d6b0870b53799204515145935608cdeb4cebb95a26800b6750e48884cc5b \
    --hash=sha256:831f2df87bd3afdfc77829bc94bd997a7c212663889d56518359c827d7113b1f \
    --hash=sha256:8d1563060e77096367952fb44fca595f2b2f477156de389ce7c0ade3aef29e21 \
    --hash=sha256:69958735d5e01f7b38226a6c6e7187d72b7e4d42b6b496aca5860b611ca0c193 \
    --hash=sha256:45a7dfbf9ed8d68fd39763940591db7637cf8817c5bce1a44f7b56c97cbe211e \
    --hash=sha256:e348ccf5bc5235fc405ab19d53bec215bb373300e5523c7b476cc0da8a5e9973
pandas==1.4.0 \
    --hash=sha256:de62cf699122dcef175988f0714678e59c453dc234c5b47b7136bfd7641e3c8c \
    --hash=sha256:46a18572f3e1cb75db59d9461940e9ba7ee38967fa48dd58f4139197f6e32280 \
    --hash=sha256:73f7da2ccc38cc988b74e5400b430b7905db5f2c413ff215506bea034eaf832d \
    --hash=sha256:5229c95db3a907451dacebc551492db6f7d01743e49bbc862f4a6010c227d187 \
    --hash=sha256:fe454180ad31bbbe1e5d111b44443258730467f035e26b4e354655ab59405871

python inspector has the error:

pyparsing.exceptions.ParseException: Expected string_end, found '\'  (at char 197), (line:1, col:198)

Removing the "\" from the file completes without errors.

[pombredanne]

@gmathiou4 this is weird because pip-requirements-parser handles these alright.

$ cat r.txt 
numpy==1.22.1 \
    --hash=sha256:3d62d6b0870b53799204515145935608cdeb4cebb95a26800b6750e48884cc5b \
    --hash=sha256:831f2df87bd3afdfc77829bc94bd997a7c212663889d56518359c827d7113b1f \
    --hash=sha256:8d1563060e77096367952fb44fca595f2b2f477156de389ce7c0ade3aef29e21 \
    --hash=sha256:69958735d5e01f7b38226a6c6e7187d72b7e4d42b6b496aca5860b611ca0c193 \
    --hash=sha256:45a7dfbf9ed8d68fd39763940591db7637cf8817c5bce1a44f7b56c97cbe211e \
    --hash=sha256:e348ccf5bc5235fc405ab19d53bec215bb373300e5523c7b476cc0da8a5e9973
pandas==1.4.0 \
    --hash=sha256:de62cf699122dcef175988f0714678e59c453dc234c5b47b7136bfd7641e3c8c \
    --hash=sha256:46a18572f3e1cb75db59d9461940e9ba7ee38967fa48dd58f4139197f6e32280 \
    --hash=sha256:73f7da2ccc38cc988b74e5400b430b7905db5f2c413ff215506bea034eaf832d \
    --hash=sha256:5229c95db3a907451dacebc551492db6f7d01743e49bbc862f4a6010c227d187 \
    --hash=sha256:fe454180ad31bbbe1e5d111b44443258730467f035e26b4e354655ab59405871
$ pip install pip-requirements-parser
$ python
Python 3.9.10 (main, Jan 29 2022, 10:01:49) 
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from pip_requirements_parser import *
>>> reqs=list(parse_requirements("r.txt"))
>>> reqs
[<pip_requirements_parser.ParsedRequirement object at 0x7f5654d76af0>, <pip_requirements_parser.ParsedRequirement object at 0x7f5654d766d0>]

But I see that we are using packvers to parse these rather than the requirements parser... good catch:

$ python-inspector -r r.txt -p 3.9 -o linux --json -
Traceback (most recent call last):
  File "/tmp/venv/lib/python3.9/site-packages/packvers/requirements.py", line 102, in __init__
    req = REQUIREMENT.parseString(requirement_string)
  File "/tmp/venv/lib/python3.9/site-packages/pyparsing/core.py", line 1141, in parse_string
    raise exc.with_traceback(None)
pyparsing.exceptions.ParseException: Expected string_end, found '\'  (at char 14), (line:1, col:15)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/tmp/venv/lib/python3.9/site-packages/python_inspector/resolve_cli.py", line 247, in resolve_dependencies
    resolution_result: Dict = resolver_api(
  File "/tmp/venv/lib/python3.9/site-packages/python_inspector/api.py", line 263, in resolve_dependencies
    resolution, purls = resolve(
  File "/tmp/venv/lib/python3.9/site-packages/python_inspector/api.py", line 316, in resolve
    requirements = list(
  File "/tmp/venv/lib/python3.9/site-packages/python_inspector/api.py", line 382, in get_requirements_from_direct_dependencies
    req = Requirement(requirement_string=dependency.extracted_requirement)
  File "/tmp/venv/lib/python3.9/site-packages/packvers/requirements.py", line 104, in __init__
    raise InvalidRequirement(
packvers.requirements.InvalidRequirement: Parse error at "'\\\n    --'": Expected string_end

[gmathiou4]

@pombredanne your team is going to take care of fixing it , or we should book some time in the future for this fix?

[pombredanne]

@gmathiou4 your help is mucho appreciated and much welcomed!