hash *_files.xml files when creating new shards; assemble and commit this information for existing shards

ArchiveTeam / IA.BAK

We back up a lot of stuff from around the web; now it's time to back up the Internet Archive, just in case.

GNU General Public License v3.0

87 stars 22 forks source link

hash *_files.xml files when creating new shards; assemble and commit this information for existing shards #51

Open db48x opened 7 years ago

db48x commented 7 years ago

This will reduce confusion from users seeing these errors scroll by as they download things, and etc. It will also allow us to detect changes to these files over time (as new metadata is added to items, for example) and periodically update the shards to match.