harvard-lil / perma

Indelible links
408 stars 72 forks source link

Optionally target user-uploaded Perma Links during WARC -> WACZ conversion testing. #3542

Closed rebeccacremona closed 3 weeks ago

rebeccacremona commented 3 weeks ago

See ENG-959.

The invoke task for launching WARC -> WACZ benchmark testing can currently target various subsets of Perma's collection.

This PR adds another option: you can now specifically target WARCs constructed from user-uploaded images or PDFs.

Since those WARCs were made using different software than the rest of our collection, and since they have a different internal structure, they might convert differently than other WARCs: this should make that easier to spot.