Wayback Machine archiver should be run in background with up to 5 jobs

bellingcat / auto-archiver

Automatically archive links to videos, images, and social media content from Google Sheets (and more).

https://pypi.org/project/auto-archiver/

MIT License

581 stars 61 forks source link

Wayback Machine archiver should be run in background with up to 5 jobs #14

Closed loganwilliams closed 2 years ago

loganwilliams commented 2 years ago

We can have up to 5 simultaneous capture requests open with the Wayback Machine Save Page Now API. Since this is one of the most time consuming archivers, it would be good to take advantage of this.