search

jsvine / waybackpack

Download the entire Wayback Machine archive for a given URL.
MIT License
2.86k stars 192 forks source link

Status code 403 causes JSONDecodeError #31

Closed jwilk closed 4 years ago

jwilk commented 6 years ago 
$ waybackpack --list http://lcamtuf.coredump.cx/
INFO:waybackpack.session: HTTP status code: 403
Traceback (most recent call last):
  File "/home/jwilk/.local/bin/waybackpack", line 11, in <module>
    load_entry_point('waybackpack==0.3.5', 'console_scripts', 'waybackpack')()
  File "/home/jwilk/.local/lib/python3.5/site-packages/waybackpack/cli.py", line 86, in main
    collapse=args.collapse
  File "/home/jwilk/.local/lib/python3.5/site-packages/waybackpack/cdx.py", line 19, in search
    "collapse": collapse
  File "/home/jwilk/.local/lib/python3.5/site-packages/requests/models.py", line 888, in json
    self.content.decode(encoding), **kwargs
  File "/usr/lib/python3.5/json/__init__.py", line 319, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python3.5/json/decoder.py", line 339, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python3.5/json/decoder.py", line 357, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

I guess waybackpack shouldn't try to parse error pages as JSON…

jsvine commented 4 years ago

A very belated thank you for raising this issue, and apologies for taking so long to respond. Having now turned back for a bit to this library, I cannot seem to reproduce the problem. Feel free to reopen this comment if the problem arises again for you.