More Error messages needed

[christian-rli]

OMI can benefit from useful error messages.

For example:

• File doesn't match metadata description
  • column doesn't exist / has different data type / has different name
  • table has more columns than described in metadata
• JSON seems to be invalid at location X
• key name does not exist in standards, typo?
• Warning: no license has been chosen, this field will be necessary when uploading to the OEP

I would recommend creating a list of possible errors first and to then write useful messages and tests for them.

Also, maybe have a look a the license check performed on OEP (implemented by @jh-RLI ) and if it makes sense to put it into OMI or leave it there.

See #99

[christian-rli]

@henhuy I thought up a list of possible error types:

in the metadata:

• wrong structure
• other version than declared
• JSON syntax error (like missing ,}]:'")
• incorrect nesting
• key name not in schema (typo?)
• additional field (this should just show a WARNING. tolerating additional fields has been requested)
• wrong data type in a JSON field
• duplicate key/value
• null values that mess with the structure of the JSON
• missing field (maybe as WARNING? When missing ID or licence)

I have provided a metadata string for each of the errors above in a separate branch. Maybe it's useful to you. I wasn't sure what a minimal metadata string had to include, so maybe they're unnecessarily big.

Checking against data:

• metadata don't match data structure
  • missing column
  • extra column
• wrong data type in a column (Warning or error?)
• duplicate or null values in ID column

I haven't created samples for these, because I saw that you had a test data table. So I would like use that for any error sample files.

[henhuy]

Tried to implement metadata checks from above. Metadata files were VERY helpful, thx! Following findings ("x" means a test is included which checks for given issue):

• [x] wrong structure is covered by either JSON decoder (if structure is no valid JSON format) or JSONschema if structure does not fit
• [ ] other version than declared Not yet possible, as only versions >1.5.2 are implemented and there are no differences compared to 1.6.0
• handled by JSON decoder
• [x] incorrect nesting handled by JSON decoder
• [ ] key name not in schema (typo?) as we don't have required fields yet, I can not test against it. Typos are out of scope IMO. (wrong_key.json will only throw error due to additional key "titel", but will not know that this is a typo)
• [ ] additional field (this should just show a WARNING. tolerating additional fields has been requested) must be allowed in Metadata schema first, otherwise JSONschema will always throw an error
• [ ] wrong data type in a JSON field should work due to JSONschema, but not yet tested as types are all "string" or "null"
• [x] duplicate key/value added check to JSON decoder which checks for duplicate keys
• [x] null values that mess with the structure of the JSON handled by JSON decoder
• [x] missing field (maybe as WARNING? When missing ID or licence) this actually raises an error, due to JSONschema

[henhuy]

Checking against data:

• metadata don't match data structure
• missing column
• extra column
• wrong data type in a column (Warning or error?)
• duplicate or null values in ID column

Added tests for all cases. And: Wrong data type raises an error

[nesnoj]

Great, big thanks for these efforts! (I guess this will also fix #40)