search

aboutcode-org / python-inspector

Inspect Python code and PyPI package manifests. Resolve Python dependencies.
22 stars 19 forks source link

python-inspector can't handle simple requirements file #98

Open heliocastro opened 2 years ago

heliocastro commented 2 years ago

With python 3.10.8, trying to use below requirement.txt failed to find dependencies. Same backtrace could be obtained with python 3.8.15, as used below to match default python-inspector recommended version

ort@c8d70fdc61ff:~$ python-inspector -r req.txt --json-pdt source.txt
Traceback (most recent call last):
File "/opt/python/versions/3.8.15/lib/python3.8/site-packages/python_inspector/resolve_cli.py", line 238, in resolve_dependencies
resolution_result: Dict = resolver_api(
File "/opt/python/versions/3.8.15/lib/python3.8/site-packages/python_inspector/api.py", line 235, in resolve_dependencies
resolution, purls = resolve(
File "/opt/python/versions/3.8.15/lib/python3.8/site-packages/python_inspector/api.py", line 292, in resolve
resolved_dependencies, packages = get_resolved_dependencies(
File "/opt/python/versions/3.8.15/lib/python3.8/site-packages/python_inspector/api.py", line 330, in get_resolved_dependencies
resolver_results = resolver.resolve(requirements=requirements, max_rounds=max_rounds)
File "/opt/python/versions/3.8.15/lib/python3.8/site-packages/resolvelib/resolvers.py", line 481, in resolve
state = resolution.resolve(requirements, max_rounds=max_rounds)
File "/opt/python/versions/3.8.15/lib/python3.8/site-packages/resolvelib/resolvers.py", line 373, in resolve
failure_causes = self._attempt_to_pin_criterion(name)
File "/opt/python/versions/3.8.15/lib/python3.8/site-packages/resolvelib/resolvers.py", line 213, in _attempt_to_pin_criterion
criteria = self._get_updated_criteria(candidate)
File "/opt/python/versions/3.8.15/lib/python3.8/site-packages/resolvelib/resolvers.py", line 203, in _get_updated_criteria
for requirement in self._p.get_dependencies(candidate=candidate):
File "/opt/python/versions/3.8.15/lib/python3.8/site-packages/python_inspector/resolution.py", line 600, in get_dependencies
return list(self._iter_dependencies(candidate))
File "/opt/python/versions/3.8.15/lib/python3.8/site-packages/python_inspector/resolution.py", line 591, in _iter_dependencies
for r in self.get_requirements_for_package(purl=purl, candidate=candidate):
File "/opt/python/versions/3.8.15/lib/python3.8/site-packages/python_inspector/resolution.py", line 470, in get_requirements_for_package_from_pypi_simple
yield from get_requirements_from_python_manifest(
File "/opt/python/versions/3.8.15/lib/python3.8/site-packages/python_inspector/resolution.py", line 304, in get_requirements_from_python_manifest
raise Exception(
Exception: Unable to collect setup.py dependencies securely: .cache/thirdparty/extracted_sdists/conan-1.52.0/conan-1.52.0/setup.py

ort@c8d70fdc61ff:~$

requirements.txt

attrs==22.1.0
banal==1.0.6
beautifulsoup4==4.11.1
binaryornot==0.4.4
bitarray==2.6.0
boolean.py==3.8
bottle==0.12.23
CacheControl==0.12.11
cachy==0.3.0
certifi==2022.9.24
cffi==1.15.1
chardet==5.0.0
charset-normalizer==2.1.1
cleo==0.8.1
click==8.1.3
clikit==0.6.2
colorama==0.4.6
commoncode==30.0.0
conan==1.52.0
crashtest==0.3.1
cryptography==38.0.3
debian-inspector==31.0.0
distlib==0.3.6
distro==1.6.0
dparse==0.6.2
dparse2==0.6.1
extractcode==30.0.0
extractcode-7z==16.5.210531
extractcode-libarchive==3.5.1.210531
fasteners==0.18
filelock==3.8.0
fingerprints==1.0.3
ftfy==6.1.1
future==0.18.2
gemfileparser==0.8.0
html5lib==1.1
idna==3.4
importlib-metadata==5.0.0
intbitset==2.4.1
isodate==0.6.1
jaraco.classes==3.2.3
jaraco.functools==3.5.2
javaproperties==0.8.1
jeepney==0.8.0
Jinja2==3.1.2
jsonstreams==0.6.0
keyring==23.11.0
license-expression==21.6.14
lockfile==0.12.2
lxml==4.9.1
MarkupSafe==2.1.1
mercurial==6.2.3
mock==4.0.3
more-itertools==9.0.0
msgpack==1.0.4
node-semver==0.6.1
normality==2.4.0
packageurl-python==0.10.4
packaging==20.9
parameter-expansion-patched==0.3.1
pastel==0.2.1
patch==1.16
patch-ng==1.17.4
pdfminer.six==20221105
pefile==2022.5.30
pexpect==4.8.0
pip-requirements-parser==31.2.0
pipenv==2022.9.24
pkginfo==1.8.3
pkginfo2==30.0.0
platformdirs==2.5.2
pluggy==0.13.1
pluginbase==1.0.1
plugincode==21.1.21
ply==3.11
poetry==1.1.13
poetry-core==1.0.8
ptyprocess==0.7.0
publicsuffix2==2.20191221
pyahocorasick==1.4.4
pycparser==2.21
pygmars==0.7.0
Pygments==2.13.0
PyJWT==2.6.0
pylev==1.4.0
pymaven-patch==0.3.0
pyparsing==3.0.9
python-dateutil==2.8.2
python-inspector==0.9.0
PyYAML==6.0
rdflib==6.2.0
requests==2.28.1
requests-toolbelt==0.9.1
resolvelib==0.8.1
saneyaml==0.5.2
scancode-toolkit==30.1.0
SecretStorage==3.3.3
shellingham==1.5.0
six==1.16.0
soupsieve==2.3.2.post1
spdx-tools==0.7.0a3
text-unidecode==1.3
tinynetrc==1.3.1
toml==0.10.2
tomlkit==0.11.6
tqdm==4.64.1
typecode==21.6.1
typecode-libmagic==5.39.210531
urllib3==1.26.12
urlpy==0.5
virtualenv==20.16.6
virtualenv-clone==0.5.7
wcwidth==0.2.5
webencodings==0.5.1
xmltodict==0.13.0
zipp==3.10.0
TG1999 commented 2 years ago

@heliocastro it looks like from the trace you need to use --analyze-setup-py-insecurely option.

So the command would be: python-inspector -r req.txt --json-pdt source.txt --analyze-setup-py-insecurely