Add jsonschema validation for larger data sources before going through Hub build

NIAID-Data-Ecosystem / nde-crawlers

Harvesting infrastructure to collect and standardize dataset and computational tool metadata

Apache License 2.0

0 stars 0 forks source link

Open flaneuse opened 2 years ago

flaneuse commented 2 years ago

Convert ES mapping to json schema
Use jsonschema package to validate data inputs before going through hub processing.
Add to testing suite (#14)

flaneuse commented 2 years ago

Ideally, should:

document the relationships to other schemas, e.g. input is derived from bioschemas:input -- I forgot the formal name for tagging this relationship. it's like sameAs, but I don't think that's the right name.
Should be maintained in a single place, so that updating in one place also updates the ES mapping file, the .json-schema validation source, the documentation on data.niaid.nih.gov, and the Data Discovery Engine schemas