search

aboutcode-org / python-inspector

Inspect Python code and PyPI package manifests. Resolve Python dependencies.
22 stars 19 forks source link

--index-url not parsed correctly #189

Open bennati opened 3 months ago

bennati commented 3 months ago

Analyzing a requirements.txt file that contains -i URL or --index-url URL fails with the error below. This was tested with 0.12.1 and 0.10.0, the parsing happens correctly in version 0.10.0. Replacing -i with --extra-index-url solves the issue.

Error:

environment: Environment(python_version='311', operating_system='linux')
repos:
 PypiSimpleRepository(index_url='https://pypi.org/simple', credentials=None)
 PypiSimpleRepository(index_url='h', credentials=None)
 PypiSimpleRepository(index_url='t', credentials=None)
 PypiSimpleRepository(index_url='t', credentials=None)
 PypiSimpleRepository(index_url='p', credentials=None)
 PypiSimpleRepository(index_url='s', credentials=None)
 PypiSimpleRepository(index_url=':', credentials=None)
 PypiSimpleRepository(index_url='', credentials=None)
 PypiSimpleRepository(index_url='', credentials=None)
 PypiSimpleRepository(index_url='a', credentials=None)
... (the remaining spells out each character of the provided URL)
Traceback (most recent call last):
  File "/Users/bennati/.pyenv/versions/scancode/lib/python3.8/site-packages/python_inspector/resolve_cli.py", line 259, in resolve_dependencies
    resolution_result: Dict = resolver_api(
  File "/Users/bennati/.pyenv/versions/scancode/lib/python3.8/site-packages/python_inspector/api.py", line 289, in resolve_dependencies
    resolution, purls = resolve(
  File "/Users/bennati/.pyenv/versions/scancode/lib/python3.8/site-packages/python_inspector/api.py", line 353, in resolve
    resolved_dependencies, packages = get_resolved_dependencies(
  File "/Users/bennati/.pyenv/versions/scancode/lib/python3.8/site-packages/python_inspector/api.py", line 394, in get_resolved_dependencies
    resolver_results = resolver.resolve(
  File "/Users/bennati/.pyenv/versions/scancode/lib/python3.8/site-packages/resolvelib/resolvers.py", line 546, in resolve
    state = resolution.resolve(requirements, max_rounds=max_rounds)
  File "/Users/bennati/.pyenv/versions/scancode/lib/python3.8/site-packages/resolvelib/resolvers.py", line 397, in resolve
    self._add_to_criteria(self.state.criteria, r, parent=None)
  File "/Users/bennati/.pyenv/versions/scancode/lib/python3.8/site-packages/resolvelib/resolvers.py", line 148, in _add_to_criteria
    matches = self._p.find_matches(
  File "/Users/bennati/.pyenv/versions/scancode/lib/python3.8/site-packages/python_inspector/resolution.py", line 621, in find_matches
    candidates = sorted(
  File "/Users/bennati/.pyenv/versions/scancode/lib/python3.8/site-packages/python_inspector/resolution.py", line 597, in _iter_matches
    self.get_versions_for_package(name=name, repo=repo))
  File "/Users/bennati/.pyenv/versions/scancode/lib/python3.8/site-packages/python_inspector/resolution.py", line 401, in get_versions_for_package
    return self.get_versions_for_package_from_repo(name, repo)
  File "/Users/bennati/.pyenv/versions/scancode/lib/python3.8/site-packages/python_inspector/resolution.py", line 412, in get_versions_for_package_from_repo
    for version, package in repo.get_package_versions(name).items():
  File "/Users/bennati/.pyenv/versions/scancode/lib/python3.8/site-packages/python_inspector/utils_pypi.py", line 1550, in get_package_versions
    self._get_package_versions_map(
  File "/Users/bennati/.pyenv/versions/scancode/lib/python3.8/site-packages/python_inspector/utils_pypi.py", line 1518, in _get_package_versions_map
    links = self.fetch_links(
  File "/Users/bennati/.pyenv/versions/scancode/lib/python3.8/site-packages/python_inspector/utils_pypi.py", line 1596, in fetch_links
    text = CACHE.get(
  File "/Users/bennati/.pyenv/versions/scancode/lib/python3.8/site-packages/python_inspector/utils_pypi.py", line 1687, in get
    content = get_file_content(
  File "/Users/bennati/.pyenv/versions/scancode/lib/python3.8/site-packages/python_inspector/utils_pypi.py", line 1736, in get_file_content
    raise Exception(f"Unsupported URL scheme: {path_or_url}")
Exception: Unsupported URL scheme: h/aiofiles
sschuberth commented 2 months ago

Looks like a duplicate of https://github.com/aboutcode-org/python-inspector/issues/173.