search

chaoss / augur

Python library and web service for Open Source Software Health and Sustainability metrics & data collection. You can find our documentation and new contributor information easily here: https://oss-augur.readthedocs.io/en/main/ and learn more about Augur at our website https://augurlabs.io
https://oss-augur.readthedocs.io/en/main/
MIT License
589 stars 845 forks source link

Message and Event dead link handling (`dev branch`) #2899

Open sgoggins opened 3 months ago

sgoggins commented 3 months ago

For different reasons there are sometimes links for messages or events associated with Issues and PRs that do not exist. Sometimes it is because the issue is deleted or issues have been disabled. Sometimes there are no messages. Here is an example error message for issue events: 

Traceback (most recent call last):
  File "/home/ubuntu/github/virtualenvs/hosted/lib/python3.11/site-packages/celery/app/trace.py", line 451, in trace_task
    R = retval = fun(*args, **kwargs)
                 ^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/github/virtualenvs/hosted/lib/python3.11/site-packages/celery/app/trace.py", line 734, in __protected_call__
    return self.run(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/github/augur/augur/tasks/github/events.py", line 39, in collect_events
    for event in event_generator:
  File "/home/ubuntu/github/augur/augur/tasks/github/events.py", line 103, in collect_pr_and_issues_events_by_number
    yield from github_data_access.paginate_resource(event_url)
  File "/home/ubuntu/github/augur/augur/tasks/github/util/github_data_access.py", line 44, in paginate_resource
    response = self.make_request_with_retries(url)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/github/augur/augur/tasks/github/util/github_data_access.py", line 114, in make_request_with_retries
    return self.__make_request_with_retries(url, method, timeout)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/github/virtualenvs/hosted/lib/python3.11/site-packages/tenacity/__init__.py", line 330, in wrapped_f
    return self(f, *args, **kw)
           ^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/github/virtualenvs/hosted/lib/python3.11/site-packages/tenacity/__init__.py", line 467, in __call__
    do = self.iter(retry_state=retry_state)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/github/virtualenvs/hosted/lib/python3.11/site-packages/tenacity/__init__.py", line 368, in iter
    result = action(retry_state)
             ^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/github/virtualenvs/hosted/lib/python3.11/site-packages/tenacity/__init__.py", line 390, in <lambda>
    self._add_action_func(lambda rs: rs.outcome.result())
                                     ^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/concurrent/futures/_base.py", line 449, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/home/ubuntu/github/virtualenvs/hosted/lib/python3.11/site-packages/tenacity/__init__.py", line 470, in __call__
    result = fn(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/github/augur/augur/tasks/github/util/github_data_access.py", line 128, in __make_request_with_retries
    return self.make_request(url, method, timeout)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/github/augur/augur/tasks/github/util/github_data_access.py", line 102, in make_request
    raise UrlNotFoundException(f"Could not find {url}")
augur.tasks.github.util.github_data_access.UrlNotFoundException: Could not find https://api.github.com/repos/dotnet/runtime/issues/92433/events
sgoggins commented 3 months ago

I think this related to issue transfer functionality that goes wrong:

https://github.com/chaoss/augur/issues/2896 Was the first:

Then we transferred it to:

https://github.com/chaoss/augur-community-reports/issues/63

Then we transferred it back and a new issue was opened:

https://github.com/chaoss/augur/issues/2899