search

4pr0n / ripme

Downloads albums in bulk
MIT License
913 stars 205 forks source link

Error: Read Timed Out when trying to download large imgur album #445

Open 9yorgos opened 7 years ago

9yorgos commented 7 years ago

just installed ripme a couple minutes to download a big imgur album and its not working. If anyone can help me out that would be awesome!

cyian-1756 commented 7 years ago

Can you post the link to the album?

9yorgos commented 7 years ago

http://imgur.com/a/7zL7T

cyian-1756 commented 7 years ago

Imgur seems to be down right now

9yorgos commented 7 years ago

really? im viewing the album as im typing this.

cyian-1756 commented 7 years ago

@9yorgos Trying to view the album just gives me a "Imgur is over capacity!" error, so at least on my end imgur is down.

metaprime commented 7 years ago

Try http://isup.me to find out what a site looks like from another server.

metaprime commented 7 years ago

Imgur does seem to have problems with albums that are approx 1000 images and larger. How large is this album? You end up getting an over capacity page for albums that large, even when navigating manually, if you've had a fair amount of imgur traffic already. Not sure what their algorithm is for this. Ripme gets best results if imgur albums are not larger than a few hundred images.

9yorgos commented 7 years ago

yea its 1100 images. So theres no way to save it? Thats unfortunate.

metaprime commented 7 years ago

Seems to be a problem on imgur's end, so at the moment, it seems it's not possible. Nothing we can do on ripme's side. Is it possible to log in to imgur and view it? We might need to use some sort of log in via ripme mechanism to enable ripping giant albums.

9yorgos commented 7 years ago

yea i can log in and view the album

metaprime commented 7 years ago

You could get the list of URLs while logged in by using imgur's rest API endpoint for album metadata, and then use ripme's new feature: Rip a list of URLs from a file from the command line with the -f flag. (#344 by @JoshKastang)

Unfortunately, Ripme still lacks the ability to rip single images, but that's coming soon.

metaprime commented 7 years ago

Also apparently the Imgur API has changed. Now requires OAuth 2. I haven't gone through the process of setting it up but here are some resources: https://api.imgur.com/ https://api.imgur.com/oauth2 https://api.imgur.com/endpoints/album

So you might be looking into making a request for this: https://api.imgur.com/3/album/7zL7T/images

But you'll need to set up authentication or you'll get a 401.

9yorgos commented 7 years ago

im confused haha. so what should i do if i want to rip the entire album? what steps should i take.

metaprime commented 7 years ago

Use the API to get the file list, put all the files into a text file (one file per line, no extra characters), and run ripme from the command line like java -jar ripme.jar -f file.txt.