search

joshua-hull / Reddit-Image-Scraper

Perl script to download imaged hosted at imgur.com linked from a subreddit at reddit.com
25 stars 8 forks source link

Error GETting i.imgur.com link. #4

Closed joshua-hull closed 11 years ago

joshua-hull commented 12 years ago

I've gotten a couple of cases now which report as 'Error GETing http://i.imgur.com/xxxxx.jpg: Not Found at ./Reddit_Image_Scraper line 46'. In each case the image has been replace or removed. However, entering the URL into a browser yields imgur's standard "The image you are requesting does not exist or is no longer available." placeholder image.

If anyone else get this message I would appreciate you letting me know.

I get a feeling that the headers returned by the link indicate an error so WWW::Mechanize just chokes and dies. For now I've disabled autocheck in the $mech instance and the script just keeps chugging along without actually writing the image to disk. If anyone can find a problem with this or a better solution I would appreciate it.

aggrolite commented 12 years ago

This should at least suppress any warnings and keep your crawler from crashing: https://github.com/joshua-hull/Reddit-Image-Scraper/pull/7