incomplete images are ending up in the gallery

teracow / googliser

a fast BASH multiple-image downloader

GNU General Public License v3.0

207 stars 38 forks source link

incomplete images are ending up in the gallery #53

Closed teracow closed 4 years ago

teracow commented 5 years ago

Edited title as I'm not yet sure why this is happening.

Test parameters were: -p cows -DSA -P200 -n1000

teracow commented 4 years ago

It's looking like there may be a bug in the way Wget retries downloads. It's causing the retried file to be appended to the original incomplete file.

teracow commented 4 years ago

Just found this on the Wget bugtracker: http://savannah.gnu.org/bugs/index.php?51840

Looks like the same problem. :(

teracow commented 4 years ago

Unable to fix this. Wget needs fixing instead. :(

MBtech commented 4 years ago

Well why not just rely on curl until the bug is fixed with wget?

teracow commented 4 years ago

Well why not just rely on curl until the bug is fixed with wget?

counter-offer: maybe we should prefer curl, only using wget as a fall-back (for systems without curl)? This would reduce the likelyhood of this bug messing-up downloaded images.