KeyError when --uniques-only flag is specified

jsvine / waybackpack

Download the entire Wayback Machine archive for a given URL.

MIT License

2.8k stars 189 forks source link

KeyError when --uniques-only flag is specified #81

Closed CaptSolo closed 1 month ago

CaptSolo commented 1 month ago

Program fails with KeyError when the --uniques-only flag is specified:

../waybackpack/cdx.py", line 46, in <listcomp>
    return [s for s in snapshots if int(s["dupecount"]) == 0]
KeyError: 'dupecount'

It looks like archive.org is not sending back the dupecount field even though the 'showDupeCount': 'true' parameter gets sent along with the request.

jsvine commented 1 month ago

Thanks for flagging, @CaptSolo! Have now pushed a change (available in v0.6.4) that provides a more informative error, although hopefully the API fixes the underlying problem (to render this moot).