search

aboutcode-org / python-inspector

Inspect Python code and PyPI package manifests. Resolve Python dependencies.
22 stars 19 forks source link

"Unsupported URL scheme" exception when analyzing requirements file #173

Open sschuberth opened 9 months ago

sschuberth commented 9 months ago

Using PI 0.11.0 to analyze the following requirements.txt file

-i https://pypi.org/simple
click==6.7
flask==1.0
itsdangerous==0.24
jinja2==2.10.1
markupsafe==1.1
werkzeug==0.15.3

gives

Traceback (most recent call last):
  File "/home/sebastian/.local/lib/python3.12/site-packages/python_inspector/resolve_cli.py", line 257, in resolve_dependencies
    resolution_result: Dict = resolver_api(
                              ^^^^^^^^^^^^^
  File "/home/sebastian/.local/lib/python3.12/site-packages/python_inspector/api.py", line 280, in resolve_dependencies
    resolution, purls = resolve(
                        ^^^^^^^^
  File "/home/sebastian/.local/lib/python3.12/site-packages/python_inspector/api.py", line 344, in resolve
    resolved_dependencies, packages = get_resolved_dependencies(
                                      ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/sebastian/.local/lib/python3.12/site-packages/python_inspector/api.py", line 385, in get_resolved_dependencies
    resolver_results = resolver.resolve(requirements=requirements, max_rounds=max_rounds)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/sebastian/.local/lib/python3.12/site-packages/resolvelib/resolvers.py", line 546, in resolve
    state = resolution.resolve(requirements, max_rounds=max_rounds)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/sebastian/.local/lib/python3.12/site-packages/resolvelib/resolvers.py", line 397, in resolve
    self._add_to_criteria(self.state.criteria, r, parent=None)
  File "/home/sebastian/.local/lib/python3.12/site-packages/resolvelib/resolvers.py", line 148, in _add_to_criteria
    matches = self._p.find_matches(
              ^^^^^^^^^^^^^^^^^^^^^
  File "/home/sebastian/.local/lib/python3.12/site-packages/python_inspector/resolution.py", line 612, in find_matches
    candidates = sorted(
                 ^^^^^^^
  File "/home/sebastian/.local/lib/python3.12/site-packages/python_inspector/resolution.py", line 588, in _iter_matches
    versions.extend(self.get_versions_for_package(name=name, repo=repo))
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/sebastian/.local/lib/python3.12/site-packages/python_inspector/resolution.py", line 398, in get_versions_for_package
    return self.get_versions_for_package_from_repo(name, repo)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/sebastian/.local/lib/python3.12/site-packages/python_inspector/resolution.py", line 409, in get_versions_for_package_from_repo
    for version, package in repo.get_package_versions(name).items():
                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/sebastian/.local/lib/python3.12/site-packages/python_inspector/utils_pypi.py", line 1548, in get_package_versions
    self._get_package_versions_map(
  File "/home/sebastian/.local/lib/python3.12/site-packages/python_inspector/utils_pypi.py", line 1516, in _get_package_versions_map
    links = self.fetch_links(
            ^^^^^^^^^^^^^^^^^
  File "/home/sebastian/.local/lib/python3.12/site-packages/python_inspector/utils_pypi.py", line 1594, in fetch_links
    text = CACHE.get(
           ^^^^^^^^^^
  File "/home/sebastian/.local/lib/python3.12/site-packages/python_inspector/utils_pypi.py", line 1685, in get
    content = get_file_content(
              ^^^^^^^^^^^^^^^^^
  File "/home/sebastian/.local/lib/python3.12/site-packages/python_inspector/utils_pypi.py", line 1734, in get_file_content
    raise Exception(f"Unsupported URL scheme: {path_or_url}")
Exception: Unsupported URL scheme: h/click

when run as

python-inspector --python-version 311 --operating-system linux --json-pdt /tmp/ort-PythonInspector17686323287359798582/python-inspector4677810856317566029.json --analyze-setup-py-insecurely --requirement requirements.txt

It used to work with PI 0.10.0, so this seems to be a regression.

This was discovered as part of running ORT tests.

sschuberth commented 3 months ago

Just to confirm the issue is still present in version 0.12.1, which prevents us from upgrading from version 0.10.0 in ORT.