stevenvachon / broken-link-checker

Find broken links, missing images, etc within your HTML.
MIT License
1.95k stars 302 forks source link

Gracefully Handle 429 Responses #198

Open danforbes opened 4 years ago

danforbes commented 4 years ago

Is your feature request related to a problem? Please describe. When I use blc as part of an automated build, the build fails because blc does not gracefully handle (retry) requests that result in 429 responses.

Describe the solution you'd like When blc encounters a 429 response, it should use exponential backoff to retry the 429-generating request some pre-defined (finite) number of times. This solution should probably be intelligent enough to group together requests to the same domain.

Describe alternatives you've considered Consider all 429 responses as successful; immediately retry all 429 responses...

Additional context https://github.com/substrate-developer-hub/substrate-developer-hub.github.io/issues/561