Respect GitHub Secondary Rate Limits

[mkowaliszyn-coursera]

Is your feature request related to a problem? Please describe. GitHub's secondary rate limits prevent the creation of many resources in a short time frame. For example, creating PRs is capped in a period and GitHub's limits kick in. git-xargs will report an error and failure to create a PR when this happens with a HTTP 503 error.

git-xargs should respect the rate-limiting headers that GitHub returns and pause/sleep for that amount of time before continuing. The secondary rate limiting prevents the use of git-xargs against large amounts of repos (dozens to hundreds) because of these limits. The update scripts need to add a long pause that kills any performance (i.e. 30s) and concurrency when dealing with many repos.

https://docs.github.com/en/rest/guides/best-practices-for-integrators#dealing-with-secondary-rate-limits

Describe the solution you'd like It would be great if git-xargs respected the GitHub X-RateLimit-* headers and automatically slept for that duration before continuing with the API requests.

https://docs.github.com/en/rest/overview/resources-in-the-rest-api#secondary-rate-limits

Describe alternatives you've considered The workaround is to put a fixed pause in the update script, up to 30s to avoid being throttled and having the batch job fail.

Additional context

[zackproser]

Thank you for the detailed bug report!

[zackproser]

@mkowaliszyn-coursera - if you have a free moment and are game for an experiment, would you please try building the binary from this PR and see if it improves performance with your use case? Thanks!

[mkowaliszyn-coursera]

Thanks @zackproser! I'll give it a try, though it may take a week or longer.