search

jsvine / waybackpack

Download the entire Wayback Machine archive for a given URL.
MIT License
2.81k stars 189 forks source link

Short sleep on 503 #3

Closed wumpus closed 8 years ago

wumpus commented 8 years ago

IA sends 503s if their servers are overloaded. If I understand your code correctly, you'll continue rapidly sending requests in that case. Better to sleep for a second. It's probably an even better idea to do the sleep for any 5xx return code.

jsvine commented 8 years ago

Thanks, @wumpus!