Open sbliven opened 8 months ago
Before implementing this we would have to agree on the schema technology and the spellings for the terms. I think that the schema language is not a major barrier to adoption, as there are very good tools for converting both schemas and data between different serializations (eg with LinkML).
One option would be to add a json schema to the dataset model. Most people I've discussed this with seem to expect the schema to apply only to the scientificMetadata, not to the whole json.
{
"pid": "123...",
"schema": "https://osc-em.github.com/oscem_schema.json",
"scientificMetadata": {
[validate this]
}
}
If "schema" is omited we would default to no validation for backwards compatibility.
JSON-LD also has good support and is used in many metadata containers (eg RO-Crate). One nice feature is that the context (aka schema) can be either a shared schema included by URI or can be inlined into the data itself. We could adapt the dataset model so that it's valid JSON-LD by including a @context
key linking a published minimal schema for scicat. This could even be overridden within scientificMetadata
to specify a more restrictive schema.
Would it make sense to validate LinkML directly?
Notes from SciCatCon day 2 discussion:
Notes from the discussion on 4/7/2023
Next steps (independent):
@context
to the response of 'Dataset - GET` endpoint (one option: add to DatasetClass so that it is stored in Mongo and part of the response)@context
as one attribute to ScientificMetadata for now to specify the schema of ScientificMetadataGET
endpointsReleated discussion: https://github.com/SciCatProject/scicat-backend-next/discussions/1308
Summary
Enable validation of scientificMetadata against a schema. The schema should be configurable for each site and default to no validation. In some cases multiple schema might be desirable, with different features available depending on the schema.
Motivation
Current Behaviour
Validation must be performed out-of-band by the ingestor tools/libraries
Expected Behaviour
isValid
endpoints check this schemaQuestions
scientificMetadata.@context
(JSON-LD style).