nexB / scancode-toolkit

:mag: ScanCode detects licenses, copyrights, dependencies by "scanning code" ... to discover and inventory open source and third-party packages used in your code. Sponsored by NLnet project https://nlnet.nl/project/vulnerabilitydatabase, the Google Summer of Code, Azure credits, nexB and others generous sponsors!
https://github.com/nexB/scancode-toolkit/releases/
2.02k stars 533 forks source link

500 when using PurlDB collect endpoint on GitHub PURL #3816

Open pombredanne opened 2 weeks ago

pombredanne commented 2 weeks ago

With http://127.0.0.1:8001/api/collect/?purl=pkg:github/llvm/llvm-project@10.0.0&addon_pipelines=collect_strings_gettext

ERROR Internal Server Error: /api/collect/
Traceback (most recent call last):
  File "/tmp/purldb/venv/lib/python3.10/site-packages/django/core/handlers/exception.py", line 55, in inner
    response = get_response(request)
  File "/tmp/purldb/venv/lib/python3.10/site-packages/django/core/handlers/base.py", line 197, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/home/pombreda/.pyenv/versions/3.10.13/lib/python3.10/contextlib.py", line 79, in inner
    return func(*args, **kwds)
  File "/tmp/purldb/venv/lib/python3.10/site-packages/django/views/decorators/csrf.py", line 65, in _view_wrapper
    return view_func(request, *args, **kwargs)
  File "/tmp/purldb/venv/lib/python3.10/site-packages/rest_framework/viewsets.py", line 124, in view
    return self.dispatch(request, *args, **kwargs)
  File "/tmp/purldb/venv/lib/python3.10/site-packages/rest_framework/views.py", line 509, in dispatch
    response = self.handle_exception(exc)
  File "/tmp/purldb/venv/lib/python3.10/site-packages/rest_framework/views.py", line 469, in handle_exception
    self.raise_uncaught_exception(exc)
  File "/tmp/purldb/venv/lib/python3.10/site-packages/rest_framework/views.py", line 480, in raise_uncaught_exception
    raise exc
  File "/tmp/purldb/venv/lib/python3.10/site-packages/rest_framework/views.py", line 506, in dispatch
    response = handler(request, *args, **kwargs)
  File "/tmp/purldb/packagedb/api.py", line 817, in list
    errors = priority_router.process(purl, **kwargs)
  File "/tmp/purldb/minecode/route.py", line 164, in process
    return endpoint(string, *args, **kwargs)
  File "/tmp/purldb/minecode/visitors/github.py", line 207, in process_request_dir_listed
    error_msg = map_fetchcode_supported_package(package_url, pipelines)
  File "/tmp/purldb/minecode/visitors/generic.py", line 108, in map_fetchcode_supported_package
    packages = [p for p in info(str(package_url)) or []]
  File "/tmp/purldb/minecode/visitors/generic.py", line 108, in <listcomp>
    packages = [p for p in info(str(package_url)) or []]
  File "/tmp/purldb/venv/lib/python3.10/site-packages/fetchcode/package_util.py", line 80, in get_package_info
    yield from get_github_packages(
  File "/tmp/purldb/venv/lib/python3.10/site-packages/fetchcode/package_util.py", line 92, in get_github_packages
    for package in _get_github_packages(
  File "/tmp/purldb/venv/lib/python3.10/site-packages/fetchcode/package_util.py", line 114, in _get_github_packages
    for tag, date in utils.fetch_github_tags_gql(purl):
  File "/tmp/purldb/venv/lib/python3.10/site-packages/fetchcode/utils.py", line 27, in fetch_github_tags_gql
    for node in fetch_github_tag_nodes(purl):
  File "/tmp/purldb/venv/lib/python3.10/site-packages/fetchcode/utils.py", line 100, in fetch_github_tag_nodes
    response = github_response(graphql_query)
  File "/tmp/purldb/venv/lib/python3.10/site-packages/fetchcode/utils.py", line 139, in github_response
    raise GitHubTokenError(msg)
fetchcode.utils.GitHubTokenError: GitHub API Token Not Set
Set your GitHub token in the GH_TOKEN environment variable.
ERROR "GET /api/collect/?purl=pkg:github/llvm/llvm-project@10.0.0&addon_pipelines=collect_strings_gettext HTTP/1.1" 500 145

but as an API I just see a 500! I should get some help instead