search

SpottedSqueak / FA-Gallery-Downloader

Dead simple gallery download for FurAffinity
MIT License
45 stars 5 forks source link

Add automatic retry after X when encountering an "FA maybe down" error? #18

Closed Nomaxice closed 3 months ago

Nomaxice commented 3 months ago

The addition of the batched downloads of multiple galleries is great, though there is an occasional issue I encounter on larger galleries or multiple ones at once, where FA will return a 520, perhaps hitting a rate limit or having a temporary hiccup, and so the program halts, this is not ideal when trying to leave it running overnight to let it go through the list of galleries selected.

Perhaps an option to set up a retry after 10s or 30s or something could be useful for this?

Nomaxice commented 3 months ago

Occured to me, this could also help with the occasional timeout that may be from a similar issue, though I think I only saw it timeout on swf files, so that may be an unrelated issue of swf files just not playing nice with the program. *Found a timeout for another file format, just now, nevermind, retry could help here then!

SpottedSqueak commented 3 months ago

Should mostly be fixed in the latest update!

Nomaxice commented 3 months ago

Seems to have worked, had two hiccups on two different files, and it retried successfully! Is there a cooldown or is it just "first time is 30sec, second fail is 60sec" regardless of how long between both fails?/if it's different or identical files?

SpottedSqueak commented 3 months ago

So long as it determines FA is actually up, it'll retry about 5 times, with an increasing amount of backoff every time till it works. So it'll do 30 to start, then 60, then 90, etc... This helps it get around the rate limiting too.

If it works, I'll mark it as done!

Nomaxice commented 3 months ago

Aye, and it resets how long it tries when it succeeds again? Asking as the log said 30sec on the first URL, then succeeded until a while later, but while the second URL was not the same, the log reported 60sec

If it does not reset on success, it might still slowly pull to 5 tries with the occasional hiccup! (they can happen a lot at times for some reason), still, this does help tremendously given that it has been thousands of files so far and it's still going strong from my test!

SpottedSqueak commented 3 months ago

Oh shoot, I'll check that out, thanks! I was sure it did reset.

SpottedSqueak commented 3 months ago

Yep, forgot to reset the retry count after a success! That should be fixed now!