4pr0n / rip

album ripper
GNU General Public License v2.0
45 stars 18 forks source link

Incomplete album downloads off deviantart #14

Closed Jin18 closed 11 years ago

Jin18 commented 11 years ago

When trying to download an album off deviantart a lot of images in the zip end up as 1 kb broken files.

4pr0n commented 11 years ago

Do you have any of the 1kb broken files? Can you rename the file to .txt and paste the contents of the file here? It might be useful for debugging.

Jin18 commented 11 years ago

Hi, sorry it took so long here's what's inside the file when opened in notepad, it seems like newer deviantart images have a different file path. The downloaded files are redirects instead of the full image. When trying to download the image from the old file path it downloads a thumbnail instead of the full file. This only happens to images uploaded to deviantart after may-ish of this year.

I removed all the "/<" and "/>"

//html head title Redirection /title /head body script type="text/javascript" window.location.href="http:\/\/yes-i-did.deviantart.com\/art\/Sock-it-to-Veronica-386189312" script noscript a href "http://yes-i-did.deviantart.com/art/Sock-it-to-Veronica-386189312" Click here to continue. a /noscript /body /html

4pr0n commented 11 years ago

Ahh ok.

Added some retry logic and attempted to work around the issue.

https://github.com/4pr0n/rip/commit/4758ef33973dfcac2c0ac0c9fcc6ba3d6bbf3a5f

It pauses for 3 seconds if it hits a redirect page before retrying (I think it has to do with rate-limits). I was able to (slowly) grab the first 72 pics from my desktop.

op pls confirm

Jin18 commented 11 years ago

Hey I tried it with a few of the links that didn't work before and they all work now. Thanks!