search

RipMeApp / ripme

Downloads albums in bulk
MIT License
3.72k stars 630 forks source link

Is RipMe not intended to download all files in a sub-reddit? #1042

Open Zachary24 opened 5 years ago

Zachary24 commented 5 years ago

Expected Behavior

Download all images (in a subreddit) from now till' beginning of the sub.

Actual Behavior

Noticed some images/videos from the sub weren't ripped, re-ripped, still nothing... Used PSAW to grab all imgur/gfycat files from the sub, then used wget to download them, a little of 25,000 images/videos, compared to ripme which only ripped about 9,000 files.. I plan to do some more testing with other subs.

cyian-1756 commented 5 years ago

Is RipMe not intended to download all files in a sub-reddit?

It is intended to download all files

Hrxn commented 5 years ago

Noticed some images/videos from the sub weren't ripped, re-ripped, still nothing... Used PSAW to grab all imgur/gfycat files from the sub, then used wget to download them, a little of 25,000 images/videos, compared to ripme which only ripped about 9,000 files.. I plan to do some more testing with other subs.

What do you mean with PSAW? This? https://github.com/dmarx/psaw

Zachary24 commented 5 years ago

Yes, PSAW connects to the PushShiftAPI which then connects to Reddit, puts all the urls in a csv, wget downloads them, then the script would delete any files that aren't, .jpg .jpeg .mp4 .gif .png..

Zachary24 commented 5 years ago

Are you sure? the script itself doesn't store the download history, doesnt automatically remove the other csv rows as well as downloads some random .html files (As it was quickly made just to test it). After seeing the difference in ripme and the basic script I decided to start building my own "ripme" alternative, which right now supports fully supports Reddit, Instagram, Wget, and youtube-dl.. Somewhat supports Imgur albums and Wikipedia Tables, and more to come! So if you're interested in that I could let you know when I'm finished.

Zachary24 commented 5 years ago

I'm very interested! Please lmk!

https://github.com/Zachary24/RipPy

Its not been released yet, its just the github page. It's almost finished though, all I really have left until the release is creating the basic google search image ripper, and cleaning up instagram youtube and reddit!

ealgase commented 5 years ago

Ripme only rips 1000 entries which is what the Reddit api returns.

jshu-git commented 5 years ago

I'm very interested! Please lmk!

https://github.com/Zachary24/RipPy

Its not been released yet, its just the github page. It's almost finished though, all I really have left until the release is creating the basic google search image ripper, and cleaning up instagram youtube and reddit!

Hey, did you ever finish RipPy? Would be very interested in trying it out as an alternative to RipMe.