joshua-hull / Reddit-Image-Scraper

Perl script to download imaged hosted at imgur.com linked from a subreddit at reddit.com
25 stars 8 forks source link

Download new #15

Closed mgsk closed 11 years ago

mgsk commented 11 years ago

It would be useful if this script could download just those pictures that haven't been downloaded before.

aggrolite commented 11 years ago

:+1: I can take a look at this unless @joshua-hull is interested

mgsk commented 11 years ago

aggrolite, I made a small change but haven't forked the repo:

    print 'Extracted ' . @links . " images for download from r/$sub\n";
    foreach my $img (@links) {
        unless (-e "$sub/" . $img->{file_name}) {
            print 'Downloading ' . $img->{url} . "\n";
            $mech->get($img->{url}, ':content_file' => "$sub/" . $img->{file_name});
            sleep INTERVAL;
        }
        else {
            print "File already exists... " . $img->{file_name} . "\n";
        }
mgsk commented 11 years ago

Sorry, didn't mean to close the issue.

joshua-hull commented 11 years ago

Added @mgsk 's code and pushed. Thanks for the suggestion. I bet imgur will appreciate us not chewing through unnecessary bandwidth.

aggrolite commented 11 years ago

:+1: