4pr0n / ripme

Downloads albums in bulk
MIT License
917 stars 203 forks source link

Allow ripping from sites that require username/password #47

Open 4pr0n opened 10 years ago

4pr0n commented 10 years ago

Examples: Facebook, Twitter, Instagram, Tumblr

These sites have some content that's only accessible from specific accounts.

4pr0n commented 10 years ago
  1. Check if album is publicly visible; if not, ask for username/password in a pop-up window
  2. Login, save cookies.
  3. Rip normally.

Most sites use OAuth2 for authentication which would be a pain point.

Phenrei commented 10 years ago

A suggestion regarding this that I've seen done by an expensive paid ripper (it would likely also fix the issues with sites you can't log in to like flickr): import the active cookies from the browser when downloading from sites. I believe Firefox would require just sqlite to do this reading, though the ripper I saw used all 4 major browsers.

Doleman commented 9 years ago

Furaffinity and Hentai-Foundry I've found don't save all the images even after the suggestion made about saving the cookie. It's a strange case because for some users it'll download all the images while for others it won't. I think it has something to do with Furaffinity's option to switch between Safe for Work and Not Safe for Work. for comparrison: lionalliance will have all of their images download even the NSFW ones (over 400 images) While giantgrowingvixen will only download 17, (when the user has over 120 images)

Phenrei commented 9 years ago

It's actually a secondary problem I coded a fix for that's pending (https://github.com/4pr0n/ripme/pull/100). Those sites support multiple image formats for files and the ripper currently assumes they're all .jpg because it doesn't go to the image page to check (like it does for some rippers) and doesn't support a check/fallback for this. Per-page checking is probably safer, but a file format fallback feature was faster to rip and less likely to get some part of the ripper banned from hitting the site too hard.