danielfett / advancedcaching

Advanced Geocaching Tool for Linux
http://www.danielfett.de/internet-und-opensource,software,agtl
31 stars 13 forks source link

HTTP404 causes too much of logs #163

Closed aapo closed 11 years ago

aapo commented 11 years ago

When cache page contains deadlink (404) to the image, it causes unnecessary massive amount of logs and even Traceback, e.g.:

139321       INFO downloader           Sending request to http://www.geocaching.com/seek/cache_details.aspx?wp=GCMEJY // downloader.py:122
145577      DEBUG downloader           Got gzip encoded answer // downloader.py:154
145580      DEBUG cachedownloader      Start parsing, pmin = 5.000000, pmax = 6.000000. // cachedownloader.py:358
145810       INFO cachedownloader      No website link found, skipping. // cachedownloader.py:387
146027      DEBUG core                 Progress: 5.200000 of 35.000000 // core.py:893
$  14% Fetching logs...
146031       INFO downloader           Sending request to http://www.geocaching.com/seek/geocache.logbook?tkn=&idx=1&num=10&decrypt=true // downloader.py:122
147454      DEBUG downloader           Got unencoded answer // downloader.py:157
147465      DEBUG cachedownloader      Start json logs parsing // cachedownloader.py:929
147476      DEBUG cachedownloader      Read 0 log entries // cachedownloader.py:946
147741      DEBUG core                 Progress: 5.500000 of 35.000000 // core.py:893
$  15% Images (1/2)...
147744       INFO cachedownloader      Downloading http://img.geocaching.com/cache/large/c97f9bde-6772-409a-9e9b-a733fecef3bd.jpg to /media/mmc1//geocaches/GCMEJY-image0.jpg // cachedownloader.py:583
147747       INFO downloader           Sending request to http://img.geocaching.com/cache/large/c97f9bde-6772-409a-9e9b-a733fecef3bd.jpg // downloader.py:122
148488      DEBUG downloader           Got unencoded answer // downloader.py:157
150921      DEBUG core                 Progress: 5.750000 of 35.000000 // core.py:893
$  16% Images (2/2)...
150935       INFO cachedownloader      Downloading http://counter.foocounter.com/counter/1108664710/miniscu to /media/mmc1//geocaches/GCMEJY-image1.img // cachedownloader.py:583
150953       INFO downloader           Sending request to http://counter.foocounter.com/counter/1108664710/miniscu // downloader.py:122
152136      ERROR cachedownloader      HTTP Error 404: Not Found // cachedownloader.py:591
Traceback (most recent call last):
  File "/opt/agtl-maemo/cachedownloader.py", line 588, in _parse_cache_page
f.write(self.downloader.get_reader(url, login = False).read())
  File "/opt/agtl-maemo/downloader.py", line 129, in get_reader
resp = urlopen(req)
  File "/usr/lib/python2.5/urllib2.py", line 124, in urlopen
return _opener.open(url, data)
  File "/usr/lib/python2.5/urllib2.py", line 387, in open
response = meth(req, response)
  File "/usr/lib/python2.5/urllib2.py", line 498, in http_response
'http', request, response, code, msg, hdrs)
  File "/usr/lib/python2.5/urllib2.py", line 425, in error
return self._call_chain(*args)
  File "/usr/lib/python2.5/urllib2.py", line 360, in _call_chain
result = func(*args)
  File "/usr/lib/python2.5/urllib2.py", line 506, in http_error_default
raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
HTTPError: HTTP Error 404: Not Found
152179      ERROR cachedownloader      Failed to download image from URL http://counter.foocounter.com/counter/1108664710/miniscu // cachedownloader.py:592