search

creativecommons / ccos-scripts

Scripts used to maintain various pieces of CC's open source presence.
MIT License
14 stars 19 forks source link

[Bug] Implement GitHub retries for 5xx status codes #179

Closed TimidRobot closed 1 year ago

TimidRobot commented 1 year ago

Description

Implement GitHub retries for 5xx status codes

Current solution

EnricoMi commented 2023-01-21 · API call returns 502 server error intermittently · Issue #2400 · PyGithub/PyGithub:

The 502 error is on GitHub's end, and as you say it is transient. It can happen any time. Retrying your request should be the best strategy here.

Provide a urllib3.util.retry.Retry instance to PyGithub, that is configured to retry 5xx responses on all types of requests:

from urllib3.util.retry import Retry
g = Github("access_token", retry=Retry(status_forcelist=list(range(500, 600)), allowed_methods={'DELETE', 'GET', 'HEAD', 'OPTIONS', ,'POST', 'PUT', 'TRACE'}))

Then PyGithub will retry requests that fail with 5xx errors for you.

https://urllib3.readthedocs.io/en/stable/reference/urllib3.util.html

Future solution (v2)

In the future, the default value of the retry parameter will be better. The code necessary has been merged:

However, at the time of writing, the code has not yet been released to stable (see Release v2.0.0-preview · PyGithub/PyGithub).

Reproduction

See occasional errors in GitHub Actions logs

move_closed_issues.py:

<module>: Unhandled exception: Traceback (most recent call last):
  File "/home/runner/work/ccos-scripts/ccos-scripts/./move_closed_issues.py", line 101, in <module>
    main()
  File "/home/runner/work/ccos-scripts/ccos-scripts/./move_closed_issues.py", line 96, in main
    move_cards(args, github, backlog, done)
  File "/home/runner/work/ccos-scripts/ccos-scripts/./move_closed_issues.py", line 70, in move_cards
    content = card.get_content(content_type="Issue")
  File "/opt/hostedtoolcache/Python/3.10.12/x64/lib/python3.10/site-packages/github/ProjectCard.py", line 135, in get_content
    headers, data = self._requester.requestJsonAndCheck("GET", url)
  File "/opt/hostedtoolcache/Python/3.10.12/x64/lib/python3.10/site-packages/github/Requester.py", line 442, in requestJsonAndCheck
    return self.__check(
  File "/opt/hostedtoolcache/Python/3.10.12/x64/lib/python3.10/site-packages/github/Requester.py", line 487, in __check
    raise self.__createException(status, responseHeaders, data)
github.GithubException.GithubException: 502 {"message": "Server Error"}

Expectation

Mitigate intermittent server errors so that only "serious" errors are reported.

Resolution