search

DocNow / diffengine

track changes to the news, where news is anything with an RSS feed
MIT License
177 stars 30 forks source link

handle too many redirects #16

Closed edsu closed 7 years ago

edsu commented 7 years ago

I caught this via an email from cron. It looks like some better handling of this type of error is needed?

Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/diffengine/__init__.py", line 233
, in archive
    resp = requests.get(save_url, headers={"User-Agent": UA})
  File "/usr/local/lib/python3.5/dist-packages/requests/api.py", line 70, in get
    return request('get', url, params=params, **kwargs)
  File "/usr/local/lib/python3.5/dist-packages/requests/api.py", line 56, in req
uest
    return session.request(method=method, url=url, **kwargs)
  File "/usr/local/lib/python3.5/dist-packages/requests/sessions.py", line 488,
in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.5/dist-packages/requests/sessions.py", line 630,
in send
    history = [resp for resp in gen] if allow_redirects else []
  File "/usr/local/lib/python3.5/dist-packages/requests/sessions.py", line 630,
in <listcomp>
    history = [resp for resp in gen] if allow_redirects else []
  File "/usr/local/lib/python3.5/dist-packages/requests/sessions.py", line 111,
in resolve_redirects
    raise TooManyRedirects('Exceeded %s redirects.' % self.max_redirects, respon
se=resp)
requests.exceptions.TooManyRedirects: Exceeded 30 redirects.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/bin/diffengine", line 9, in <module>
    load_entry_point('diffengine==0.0.27', 'console_scripts', 'diffengine')()
  File "/usr/local/lib/python3.5/dist-packages/diffengine/__init__.py", line 459
, in main
    version = entry.get_latest()
  File "/usr/local/lib/python3.5/dist-packages/diffengine/__init__.py", line 173
, in get_latest
    new.archive()
  File "/usr/local/lib/python3.5/dist-packages/diffengine/__init__.py", line 242
, in archive
    save_url, resp.headers, e
UnboundLocalError: local variable 'resp' referenced before assignment
edsu commented 7 years ago

It's interesting, this is coming from https://web.archive.org/save/ The fatal exception was from broken attempt at logging. Otherwise the too many redirects would have been handled. I wonder what might be going on with the save-to-internet-archive function though.