Closed aapo closed 11 years ago
When cache page contains deadlink (404) to the image, it causes unnecessary massive amount of logs and even Traceback, e.g.:
139321 INFO downloader Sending request to http://www.geocaching.com/seek/cache_details.aspx?wp=GCMEJY // downloader.py:122 145577 DEBUG downloader Got gzip encoded answer // downloader.py:154 145580 DEBUG cachedownloader Start parsing, pmin = 5.000000, pmax = 6.000000. // cachedownloader.py:358 145810 INFO cachedownloader No website link found, skipping. // cachedownloader.py:387 146027 DEBUG core Progress: 5.200000 of 35.000000 // core.py:893 $ 14% Fetching logs... 146031 INFO downloader Sending request to http://www.geocaching.com/seek/geocache.logbook?tkn=&idx=1&num=10&decrypt=true // downloader.py:122 147454 DEBUG downloader Got unencoded answer // downloader.py:157 147465 DEBUG cachedownloader Start json logs parsing // cachedownloader.py:929 147476 DEBUG cachedownloader Read 0 log entries // cachedownloader.py:946 147741 DEBUG core Progress: 5.500000 of 35.000000 // core.py:893 $ 15% Images (1/2)... 147744 INFO cachedownloader Downloading http://img.geocaching.com/cache/large/c97f9bde-6772-409a-9e9b-a733fecef3bd.jpg to /media/mmc1//geocaches/GCMEJY-image0.jpg // cachedownloader.py:583 147747 INFO downloader Sending request to http://img.geocaching.com/cache/large/c97f9bde-6772-409a-9e9b-a733fecef3bd.jpg // downloader.py:122 148488 DEBUG downloader Got unencoded answer // downloader.py:157 150921 DEBUG core Progress: 5.750000 of 35.000000 // core.py:893 $ 16% Images (2/2)... 150935 INFO cachedownloader Downloading http://counter.foocounter.com/counter/1108664710/miniscu to /media/mmc1//geocaches/GCMEJY-image1.img // cachedownloader.py:583 150953 INFO downloader Sending request to http://counter.foocounter.com/counter/1108664710/miniscu // downloader.py:122 152136 ERROR cachedownloader HTTP Error 404: Not Found // cachedownloader.py:591 Traceback (most recent call last): File "/opt/agtl-maemo/cachedownloader.py", line 588, in _parse_cache_page f.write(self.downloader.get_reader(url, login = False).read()) File "/opt/agtl-maemo/downloader.py", line 129, in get_reader resp = urlopen(req) File "/usr/lib/python2.5/urllib2.py", line 124, in urlopen return _opener.open(url, data) File "/usr/lib/python2.5/urllib2.py", line 387, in open response = meth(req, response) File "/usr/lib/python2.5/urllib2.py", line 498, in http_response 'http', request, response, code, msg, hdrs) File "/usr/lib/python2.5/urllib2.py", line 425, in error return self._call_chain(*args) File "/usr/lib/python2.5/urllib2.py", line 360, in _call_chain result = func(*args) File "/usr/lib/python2.5/urllib2.py", line 506, in http_error_default raise HTTPError(req.get_full_url(), code, msg, hdrs, fp) HTTPError: HTTP Error 404: Not Found 152179 ERROR cachedownloader Failed to download image from URL http://counter.foocounter.com/counter/1108664710/miniscu // cachedownloader.py:592
When cache page contains deadlink (404) to the image, it causes unnecessary massive amount of logs and even Traceback, e.g.: