ArchiveTeam / imgur-grab

Archiving imgur.
The Unlicense
65 stars 3 forks source link

Reduce timeout for 429 errors? #6

Closed Leseratte10 closed 1 year ago

Leseratte10 commented 1 year ago

Since imgur apparently returns HTTP error 429 for nearly everyone trying to download MP4s, would it be a good idea to drop the retry count drastically?

Right now almost everyone is wasting time waiting for the 429 to timeout after like 10 attempts, instead of focusing on the JPGs and PNGs that can still be downloaded.

Arkiver2 commented 1 year ago

This problem was specifically for mp4s, and is handled in commit https://github.com/ArchiveTeam/imgur-grab/commit/aed938aab47fb56d4dfefe0e08791b1c60b22e1f. Therefore I'm closing this PR. Feel free to make another PR/issue if the 429s are problematic for another aspect other than bans in case of too high concurrent.