coleifer / micawber

a small library for extracting rich content from urls
http://micawber.readthedocs.org/
MIT License
635 stars 91 forks source link

Also catch HTTPErrors when opening url. #8

Closed benkonrath closed 12 years ago

benkonrath commented 12 years ago

HttpErrors float through when running with PyPy 1.9. This patch should fix the problem.

Here's a trace of the problem with youtube url:

https://www.youtube.com/watch?v=Z3IsAaLtO_4

    oembed_data = providers.request(self.url)
  File "/var/lib/jenkins/jobs/legacydb/workspace/env/src/micawber/micawber/providers.py", line 82, in inner
    return fn(self, url, **params)
  File "/var/lib/jenkins/jobs/legacydb/workspace/env/src/micawber/micawber/providers.py", line 108, in request
    return provider.request(url, **params)
  File "/var/lib/jenkins/jobs/legacydb/workspace/env/src/micawber/micawber/providers.py", line 57, in request
    response = self.fetch(endpoint_url)
  File "/var/lib/jenkins/jobs/legacydb/workspace/env/src/micawber/micawber/providers.py", line 29, in fetch
    resp = urllib2.urlopen(req)
  File "/var/lib/jenkins/jobs/legacydb/workspace/pypy-1.9/lib-python/2.7/urllib2.py", line 126, in urlopen
    return _opener.open(url, data, timeout)
  File "/var/lib/jenkins/jobs/legacydb/workspace/pypy-1.9/lib-python/2.7/urllib2.py", line 400, in open
    response = meth(req, response)
  File "/var/lib/jenkins/jobs/legacydb/workspace/pypy-1.9/lib-python/2.7/urllib2.py", line 513, in http_response
    'http', request, response, code, msg, hdrs)
  File "/var/lib/jenkins/jobs/legacydb/workspace/pypy-1.9/lib-python/2.7/urllib2.py", line 438, in error
    return self._call_chain(*args)
  File "/var/lib/jenkins/jobs/legacydb/workspace/pypy-1.9/lib-python/2.7/urllib2.py", line 372, in _call_chain
    result = func(*args)
  File "/var/lib/jenkins/jobs/legacydb/workspace/pypy-1.9/lib-python/2.7/urllib2.py", line 521, in http_error_default
    raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
HTTPError: HTTP Error 401: Unauthorized

Thanks! Ben

coleifer commented 12 years ago

Thanks for this!

benkonrath commented 12 years ago

No problem. Thanks for a great Django app. Any chance you can cut a maintenance release (0.2.5) with this fix?