Open jacg opened 2 years ago
Hashes of the configuration should be semantic hashes, rather than textual: the hash value should be a function of meaningful content of the configuration, and ignore irrelevant information such as layout, order of key-value pairs, synonyms, etc.
The MC, LOR-reconstruction and other links in our toolchain should spit out their metadata, and these should be included in the metadata of components further down the chain, which consume the data produced by earlier steps.
How about TOML for the config file syntax? Dhall seems to be the only config-related thing that does sematic hashing, but Dhall might be overkill and inconvenient for our purposes. We can probably get semantic hashing of TOML by hashing the Value
type from the Rust toml crate, after using it to parse the config file.
Keeping track of the metadata describing the parameters affecting reconstructed images, is far too painful and error-prone.
One idea is to define a consistent scheme for encoding metadata in filesystem paths and use
skim
to find matching data.However, if #15 is implemented, maybe the configuration file can serve as an adequate store of metadata. In which case, the output directory should be automatically generated from a hash of the config file's contents. Thus, no metadata would be stored in the path itself, but within files in the output directory.