Open chStaiger opened 1 month ago
Here is how I read in the toml file: https://github.com/UtrechtUniversity/autumn-fair/blob/200f234f2fb1496f195cc1fa515411c114134076/src/check_data.py#L95
And this is the code to get the information into a workable python data structure: https://github.com/UtrechtUniversity/autumn-fair/blob/200f234f2fb1496f195cc1fa515411c114134076/src/check_data.py#L12
I made a new version of metadata for validation ("_validation_schema_v2.toml") and renamed the old one (v1).
This is an example of the v2 file structure:
[hosts]
host_id.type = "string"
host_id.format = "AA0_00000"
host_groupNumber.type = "integer"
host_sex.type = "string"
host_sex.format = "A"
host_sex.values = ["M","F"]
host_age.type = "integer"
host_death.type = "integer"
host_species.type = "string"
host_breed.type = "string"
The levels are metadata["events"]["host_id"]["type"]
Not all the columns have all the possible attributes, so I added the complete list of possible attributes at the very beginning of the file (metadata_keys
). Compared to the previous version, this structure removes a layer, with the caveat that now you need to check for keywords listed in metadata_keys
Reading in the validation.toml leads to a very nested dictionary and list of dictionaries. We need to discuss how to flatten that a bit. @StefanoRapisarda can you please have a look in the code how I am using the toml and give some suggestions how to flatten the data structure a bit?