search

4pr0n / ripme

Downloads albums in bulk
MIT License
918 stars 204 forks source link

Request sites to support here #8

Open 4pr0n opened 10 years ago

4pr0n commented 10 years ago

PLEASE include an example URL of the type of album to rip.

Supported sites + commits:

Pending sites + examples (NSFW):

Enhancement requests:

CaspianRoach commented 10 years ago

wakaba threads on imageboards (such as 4chan or more examples here http://allchans.org/)

Random example: http://desuchan.net/v/res/7034.html Can probably be done by retrieving all that point to an image file extension. Or constricted to /src/ folder.

With that in mind, maybe this method can be expanded to allow the ripping of 'generic' web album pages, the ones that link directly to the files.

4pr0n commented 10 years ago

I'll look into supporting generic *chan* sites.

maybe this method can be expanded to allow the ripping of 'generic' web album pages, the ones that link directly to the files.

For that, there's already DownThemAll, the Firefox add-on which works really well.

4pr0n commented 10 years ago

maybe this method can be expanded to allow the ripping of 'generic' web album pages, the ones that link directly to the files.

I've had an idea for a ripper that can scrape anything, like random message boards that link to imagebam.com galleries or imgur albums. It would default to just try to scrape the page if the host isn't strictly supported. It would require refactoring the existing rippers, and adding a ton of one-off single-image hosts (imagetwist etc) which would be painful, but it's possible.

CaspianRoach commented 10 years ago

Not sure if it's worth your time, but there's this site: http://kinkyshare.com and I just can't leave it because I'm constantly opening new images from the bottom side :)

anyway, they sometimes have galleries like this http://kinkyshare.com/c/1857/blonde_and_brunette_margo_et_lilas_-_amiti_et_complicit and this http://kinkyshare.com/c/1384/gia_felino_-_barbie_latex_catsuit

If it's not too much trouble a ripper for albums or even tag pages here would be nice.

CaspianRoach commented 10 years ago

Pornhub seems to have a decent number of albums, too. http://www.pornhub.com/albums It feels like a popular enough site to bother with a ripper for it.

domonic46 commented 10 years ago

Request for support for: Butttoucher.com getgonewild.com i.rarchives.com user pages and teenplanet.org Thank you for making this ripper!

4pr0n commented 10 years ago

Request for support for: Butttoucher.com getgonewild.com i.rarchives.com user pages

Instead of waiting for these rippers to be implemented, try ripping http://gonewild.com/user/username (replace username with the user's name). It's more exhaustive than the above sites but doesn't always have the user.

slipl commented 10 years ago

Is Photobucket on the list?

4pr0n commented 10 years ago

Is Photobucket on the list?

It is now. Stay (patiently) tuned

4pr0n commented 10 years ago

3 commits above are in latest version & ripme should be updateable to this version.

@domonic46: getgonewild doesn't appear to work -- I can't find any "profiles" that contain imgur links. Hence the bolded text in OP:

Requests must include an example URL

slipl commented 10 years ago

Flickr support would be very nice too, thanks!

Jar4ek commented 10 years ago

Request for support for: flickr.com | For example, the photo on this page is not available download option. https://www.flickr.com/photos/beargills/

artician.com pixiv.net

Thank you for making this ripper!

crimble commented 10 years ago

Possible to add a video ripper for something like this http://vk.com/videos213696261

4pr0n commented 10 years ago

Haven't fully tested the VK video album ripper (hard to with a sample album of 2,000+ videos). It starts downloading them, so I guess it works.

Jar4ek commented 10 years ago

Recently I asked for support for Flickr page and it works very good, but there is a problem. Namely, the pictures were downloaded link only partially.

Link: https://www.flickr.com/photos/beargills/ to 1,470 images downloaded only 100 photos. I'd asked for a solution to this problem.

Next Link : www.flickr.com/photos/gestiefeltekatze to 1,290 images downloaded only 100 photos. I'd asked for a solution to this problem.

I also failed to download photos from this site:

https://www.flickr.com/photos/bubuyalove/

I also would ask for support downloading photos from the 500px.com

If there is such a possibility, please let me know the solution to this problem.

Best Regards Jarek.

crimble commented 10 years ago

The VK video ripper is working great so far but before I attempt all 2,000+ videos, are there any original file names available to replace the current output?

What I'm getting now: 001_f4fc9d1705.720.mp4 002_dc3bda4117.720.mp4 003_9bc4105e2d.720.mp4

4pr0n commented 10 years ago

Looks like Flickr changed the login scheme to use javascript to set cookies that are required for logging in. The scheme is too complex for me to bother with. This means the Flickr ripper can't rip NSFW images.

I left the code to login (which used to work a few months ago) in case someone else wants to pick it up.

ghost commented 10 years ago

Request for support for MINUS http://exogallery.minus.com/mYzoL2DVlmXMJ

erofanboy commented 10 years ago

is it possible to rip partially tumblr archives, e.g. http://***.tumblr.com/archive/2014/4 ?

jadedgnome commented 10 years ago

twitter hashtags and keyword media search results

https://twitter.com/search?q=cosplay&src=typd&mode=photos

slipl commented 10 years ago

Anon-ib.com with a log of text if possible Example: http://anon-ib.com/red/res/205.html

Fonziesthumbs commented 10 years ago

Request support for fapproved.com Example: http://fapproved.com/users/swimer210/images -or- http://fapproved.com/users/missballerina/images

The program has come a long way. Thanks for all your hard work in making it better. Take care.

domonic46 commented 10 years ago

Example for Getgonewild: http://www.getgonewild.com/profile/rollmeup I double checked for imgur links with the ripper http://176.31.245.135/rips/#getgonewild_rollmeup btw statigr.am is now iconosquare.com

domonic46 commented 10 years ago

Gifyo.com Example URL: http://gifyo.com/kittiekaulitz48/

ghost commented 10 years ago

pixiv:

  1. http://www.pixiv.net/member.php?id=2136123 HentaiFoundry:
  2. http://www.hentai-foundry.com/pictures/user/LiquidShadow
  3. http://www.hentai-foundry.com/pictures/user/LiquidShadow/scraps Facebook:
  4. https://www.facebook.com/EmanueleFerrariPhotographer/photos_stream
  5. https://www.facebook.com/EmanueleFerrariPhotographer/photos_albums Supertangas:
  6. http://www.supertangas.com/fotos/?level=search&exact=1&searchterms=Vikis%20%28Argentina%29 datwin
  7. http://datw.in/gorgeous-brunette-on-vacation/
domonic46 commented 10 years ago

Vine.co Example: https://vine.co/u/921539294726860800

metaprime commented 10 years ago

EHentai Galleries added in f9e8dd33060167a10c572495dc9d0ec8886d06ad

singhs-apps commented 10 years ago

http://drawcrowd.com http://drawcrowd.com/rabbiteyes

&

http://www.pinterest.com/

tim712 commented 10 years ago

Model Mayhem: http://www.modelmayhem.com/759568

4pr0n commented 10 years ago

Modelmayhem mostly works but can't get images that require being logged in (unless someone can provide a verified account).

elwoodflame commented 10 years ago

Would love to see a http://www.vidble.com Ripper here is a random link http://www.vidble.com/explore/o6WEglKjYW.jpg

terryinnit commented 10 years ago

Please add:

Lookbook http://lookbook.nu/mayowo

ohhdemgirls commented 10 years ago

Please support imagestash.org and smutty.com by tags

https://imagestash.org/tag/everydayuncensor http://smutty.com/h/Brunette/

ghost commented 10 years ago

4shared.com would be a good one.

Example: http://www.4shared.com/folder/pTUpT6wG/Heather.html

4pr0n commented 10 years ago

smutty.com support was added in https://github.com/4pr0n/ripme/commit/e9bee91aba119d2a5fffe50f02ace9dab3d8e6dc

ohhdemgirls commented 10 years ago

Please support imgbox.com albums.

http://imgbox.com/g/tkf1S83GI0

4pr0n commented 10 years ago

@Opzon Upgrade to 1.0.52 and you should have the option to disable popup notifications when the window is not in focus. The options should be disabled by default anyway.

MildPanda commented 10 years ago

@4pr0n thanks! Really glad to see this program get quick updates after suggestions. Would there be any possibility to add the function to remove duplicate images in the future as I'm sure that may take awhile to add? (look at exact file size and delete one)

MildPanda commented 10 years ago

my ripme frequently freezes. Idk why :( edit: happens when I rerip (I do have a good amount of reddit links to rip) Edit2: think I found the problem. for whatever reason a link got into the program history that was http://yhttp:// (more to it but you can see the error there)

ohhdemgirls commented 10 years ago

Please support mediacru.sh albums.

https://mediacru.sh/1fb54a6b5f34 https://mediacru.sh/1fb54a6b5f34.json

Note: Example to be updated, albums can contain multiple media types. webm,gif,png,jpe,mp4,mp3,etc.

Example update https://mediacru.sh/a4576fc20dcd https://mediacru.sh/a4576fc20dcd.json

elwoodflame commented 10 years ago

Is http://www.vidble.com/ still not working when you try to rip a reddit user?

On Wed, Jun 4, 2014 at 1:25 AM, ohhdemgirls notifications@github.com wrote:

Please support mediacru.sh albums.

https://mediacru.sh/1fb54a6b5f34 https://mediacru.sh/1fb54a6b5f34.json

Note: Example to be updated, albums can contain multiple media types. webm,gif,png,jpe,mp4,mp3,etc.

— Reply to this email directly or view it on GitHub https://github.com/4pr0n/ripme/issues/8#issuecomment-45063806.

4pr0n commented 10 years ago

Got mediacrush working, but there's a big problem:

Older versions of Java (likely the lion's share of users' Java) can't support encryption keys bigger than 1024 bits [Stackoverflow]. Mediacrush forces SSL & this high encryption level.

The solution for these older versions requires installing a newer version of Java or downloading and replacing files in the user's JRE (the Java Cryptography Extension (JCE) Unlimited Strength Jurisdiction Policy Files). Neither of these solutions are ideal. I'm able to catch the RuntimeException so I might just have a pop-up that says you can't rip mediacrush without upgrading to the latest Java 7 or 8.

@elwoodflame Vidble still works for me on the sample albums I found from gonewild. Please create a new issue if it's not properly ripping a URL (and please link to the problem URL in the new issue).

@Opzon

Would there be any possibility to add the function to remove duplicate images in the future

There's been requests to allow duplicate files (same URL) so this is really splitting hairs. I haven't found very many albums that contain exact duplicate files, so I don't see the payoff. If this functionality was added, it would only be able to remove duplicate files found during the same rip; previous rips/files would not be checked for dupes. Please make a new issue if you want this feature.

MildPanda commented 10 years ago

@4pr0n Easy dupe finder says I have 1k dupe files in the rip folder. I download from gonewild girls. http://gyazo.com/bcd59eba5c75c019465ae3e254bc3a0a of the dupelicate finder. (If I remove them and rerip they will just get downloaded again) Edit: don't hate on my folder names.

4pr0n commented 10 years ago

Please make a new issue if you are not requesting a new site to be supported.

@Opzon Like this: https://github.com/4pr0n/ripme/issues/55

MildPanda commented 10 years ago

Oh, alright. Sorry :( didn't understand. Edit: thanks for making it and glad to see constant improvements to the program

Jar4ek commented 10 years ago

I wonder whether it could have to RipMe also support page:

Behance : https://www.behance.net/ https://www.behance.net/HS-nj

Dribbble : https://dribbble.com/ https://dribbble.com/studioMUTI

500px : http://500px.com/ http://500px.com/jing-xin

FetLife : https://fetlife.com/ https://fetlife.com/users/2089769

Diverxity : http://www.diverxity.com/ http://www.diverxity.com/users/953

Artician : http://www.artician.com/ http://zionenciel.artician.com/

Blogspot (I'm not sure if we can get it here, but it would be nice to really like that could be done) http://www.blogspot.com/ http://marcherryfashion.blogspot.com/

ImageFap: http://www.imagefap.com/ http://www.imagefap.com/profile/SeduceMeNow666

Motherless http://motherless.com/ http://motherless.com/m/bradd

Imagearn http://imagearn.com/ http://imagearn.com/gallery.php?id=555542

So far, many links. :) How do I find any more, it certainly write about it. Yet once as Rip was formerly a party, it was a portal with images of Anime. I can not remember what it was the address, so it was a yellow portal. How would someone remembered, I would ask to give him the address.

Best Regards Jarek

4pr0n commented 10 years ago

@Jar4ek imagefap, imagearn, and motherless are already supported. I'll look into blogspot and 500px, possibly the others.

Jar4ek commented 10 years ago

@4pr0n Thank you, if you need some Grant. It write, I think I will be able to support you, but enough. :) Because your program is really great. I was looking for something like this. :)

Flip32 commented 10 years ago

hey request for Imagebam.com

ex.: http://www.imagebam.com/gallery/bo8mige1xe66j6q2g41pus2p550zvwj6/

metaprime commented 10 years ago

@4pr0n can you update the list in the original comment on this issue to reflect all implemented and all unimplemented requests. It's hard to filter through this list and see which requests are still pending.