Closed FoodProduct closed 6 years ago
When fetching a very long list of matching repos from GitHub, a consumer will run into a rate limiting error during shepherd checkout.
shepherd checkout
dbarnes:shepherd-migrations dbarnes$ shepherd checkout migrations/2018-09-cdn-url-updates/ { HttpError: { "documentation_url": "https://developer.github.com/v3/#abuse-rate-limits", "message": "You have triggered an abuse detection mechanism. Please wait a few minutes before you try again." } at response.text.then.message (/srv/nerdwallet/shepherd/node_modules/@octokit/rest/lib/request/request.js:72:19) at process._tickCallback (internal/process/next_tick.js:103:7) name: 'HttpError', code: 403, status: undefined, headers: { 'access-control-allow-origin': '*', 'access-control-expose-headers': 'ETag, Link, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval', connection: 'close', 'content-encoding': 'gzip', 'content-security-policy': 'default-src \'none\'; base-uri \'self\'; block-all-mixed-content; connect-src \'self\' uploads.github.com status.github.com collector.githubapp.com api.github.com www.google-analytics.com github-cloud.s3.amazonaws.com github-production-repository-file-5c1aeb.s3.amazonaws.com github-production-upload-manifest-file-7fdce7.s3.amazonaws.com github-production-user-asset-6210df.s3.amazonaws.com; font-src assets-cdn.github.com; form-action \'self\' github.com gist.github.com; frame-ancestors \'none\'; frame-src render.githubusercontent.com; img-src \'self\' data: assets-cdn.github.com media.githubusercontent.com camo.githubusercontent.com identicons.github.com collector.githubapp.com avatars0.githubusercontent.com avatars1.githubusercontent.com avatars2.githubusercontent.com avatars3.githubusercontent.com github-cloud.s3.amazonaws.com; manifest-src \'self\'; media-src \'none\'; script-src assets-cdn.github.com; style-src \'unsafe-inline\' assets-cdn.github.com', 'content-type': 'application/json; charset=utf-8', date: 'Mon, 24 Sep 2018 21:30:00 GMT', 'expect-ct': 'max-age=2592000, report-uri="https://api.github.com/_private/browser/errors"', 'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin', 'retry-after': '60', server: 'GitHub.com', status: '403 Forbidden', 'strict-transport-security': 'max-age=31536000; includeSubdomains; preload', 'transfer-encoding': 'chunked', 'x-content-type-options': 'nosniff', 'x-frame-options': 'deny', 'x-github-media-type': 'github.v3; format=json', 'x-github-request-id': '5A8C:0D02:256FFC7:2EBD74C:5BA95758', 'x-runtime-rack': '0.007640', 'x-xss-protection': '1; mode=block' } }
Proposal: Try/catch when fetching and obey the retry-after header returned from GitHub before continuing to paginate through the list of results. https://developer.github.com/v3/guides/best-practices-for-integrators/
retry-after
Fixed by #59.
When fetching a very long list of matching repos from GitHub, a consumer will run into a rate limiting error during
shepherd checkout
.Proposal: Try/catch when fetching and obey the
retry-after
header returned from GitHub before continuing to paginate through the list of results. https://developer.github.com/v3/guides/best-practices-for-integrators/