Flatten _validation.toml

[chStaiger]

Reading in the validation.toml leads to a very nested dictionary and list of dictionaries. We need to discuss how to flatten that a bit. @StefanoRapisarda can you please have a look in the code how I am using the toml and give some suggestions how to flatten the data structure a bit?

[chStaiger]

Here is how I read in the toml file: https://github.com/UtrechtUniversity/autumn-fair/blob/200f234f2fb1496f195cc1fa515411c114134076/src/check_data.py#L95

And this is the code to get the information into a workable python data structure: https://github.com/UtrechtUniversity/autumn-fair/blob/200f234f2fb1496f195cc1fa515411c114134076/src/check_data.py#L12

[StefanoRapisarda]

I made a new version of metadata for validation ("_validation_schema_v2.toml") and renamed the old one (v1).

This is an example of the v2 file structure:

[hosts]
host_id.type = "string"
host_id.format = "AA0_00000"
host_groupNumber.type = "integer"
host_sex.type = "string"
host_sex.format = "A"
host_sex.values = ["M","F"]
host_age.type = "integer"
host_death.type = "integer"
host_species.type = "string"
host_breed.type = "string"

The levels are --> --> . So to access the type of the host_id column in the events file you should use metadata["events"]["host_id"]["type"]

Not all the columns have all the possible attributes, so I added the complete list of possible attributes at the very beginning of the file (metadata_keys). Compared to the previous version, this structure removes a layer, with the caveat that now you need to check for keywords listed in metadata_keys