Closed themightychris closed 1 year ago
Snapshot crawls have been crashing part way through due to GitHub's new soft rate limits: https://docs.github.com/en/rest/guides/best-practices-for-integrators?apiVersion=2022-11-28#dealing-with-secondary-rate-limits
This PR implements respect for Retry-After header values after GitHub returns 403 errors (note: github does not return 429 like it should for rate limiting issues)
Retry-After
Snapshot crawls have been crashing part way through due to GitHub's new soft rate limits: https://docs.github.com/en/rest/guides/best-practices-for-integrators?apiVersion=2022-11-28#dealing-with-secondary-rate-limits
This PR implements respect for
Retry-Afterheader values after GitHub returns 403 errors (note: github does not return 429 like it should for rate limiting issues)