Engine for analysis of Siegfried export files and DROID CSV. The tool has three purposes, break the export into its components and store them within a SQLite database; create additional columns to augment the output where useful; and query the SQLite database, outputting results in a readable form useful for analysis by researchers and archivists within digital preservation departments in memory institutions. The tool will find duplicates, unidentified files, blacklisted objects, character encoding issues, and more.
The output below is roughly all the top level fields. There are at least two problems with the below:
Insensitive naming in the context of a collection. Bad File Names is not representative of both the end-user's collection and the intention of the code. We want to express something more like "names that need more care", e.g. we've all seen ���� when we don't want to. This is more about making sure data is preserved in end to end workflows.
NB. In the output report, this field is Identifying Non-ASCII and System File Names. The naming comes from the member variable within the analysis results object.
Naming conventions. The naming conventions are all over the place. Who knew PEP8 was a thing even in 2014?! (jk) At least snake case these. Golang JSON naming should be considered. In golang Capitalized fields are exported and can be read implicitly into code: bof_distance here, which is correct, might become BOFDistance, collectionsize becomes CollectionSize. I don't know if these names can be aliased somehow, where .__dict__ outputs member variables as-is.
There's a lot of data output, but JSON tools might be able to use this sensible. I should consider documenting examples.
Using the following pattern starts to get us into the realm of a decent JSON output.
The output below is roughly all the top level fields. There are at least two problems with the below:
NB. In the output report, this field is
Identifying Non-ASCII and System File Names. The naming comes from the member variable within the analysis results object.bof_distancehere, which is correct, might becomeBOFDistance,collectionsizebecomesCollectionSize. I don't know if these names can be aliased somehow, where.__dict__outputs member variables as-is.There's a lot of data output, but JSON tools might be able to use this sensible. I should consider documenting examples.
Example output: