W A R N I N G : CRUCIAL FEATURE DEPRECATED. NO LONGER WORKS.
Unfortunately, Reddit has removed the ability to scan for submissions by date range. This eliminates the usefulness of this program and I'm deeply sorry to anyone who enjoyed using it for their use. Please message Reddit's technical team so we can have it back! Click here for the official announcement: https://www.reddit.com/r/changelog/comments/7tus5f/update_to_search_api/
Thanks -2hands10fingers
Reddit Image Scraper 1.0 brought to you by pyStudyGroup
Credits: /u/2hands10fingers, /u/Joe_Anonimist, /u/iakovosbelonias, tjcim
Scrapes Reddit images from any public subreddit by a user-specified date range and saves them to your computer.
Introduction
Many tools scrape Reddit but don't give you the power to grab data on your terms. This changes things.
How it Works
Use Python 3 to scrape any .jpg or .png images in the available subreddit (you can always add or remove file extensions within the code). Simply run the program in terminal using $ python3 RedditImageScraper.py under the correct directory where RedditImageScraper.py is downloaded, enter your start and ending dates, then you will select any public subreddit of your choice, and PRESTO! you will begin downloading images.
Installation
1. Download
You can download this repository directly from GitHub and place the files where you want to run the program from. Remember, where the program is located, that is where your pictures will download to. You can change file_path in config.py to direct all downloads to that directory.
2. Install Dependencies
We make sure things are easy to install and test by using the requirements.txt file. In your terminal, type $ pip install -r requirements.txt and you should have everything you need to get going.
3. Obtaining Reddit Access
Oauth2 is required. Reddit doesn't like anonymous people taking their stuff, so you will need to copy the config.py.sample to config.py then open up the config.py file to add your own username, password, client_id, and client_secret to the file. Please follow Reddit's Oauth2 simple to follow guidelines to grab the client_id and client_secret here.
Testing
RIS now comes with testing abilities and error logging. We recommend doing this within python3's virtual environement.
Setupping up the virtual testing environment:
$ python3 -m venv env
$ source env/bin/activate
$ pip install -r requirements.txt
$ pytest
Notes
Download speeds may vary depending on the length of your selected date range, the Reddit server's current performance, or your current internet speed. You will only be able to select the years 2005 (when Reddit was launched) through the current year. If you want to download all of the images for the current day, simply enter the date twice (e.g., 3-14-2017 -- 3-14-2017).
Thank you and enjoy,
2hands10fingers
