Develop CEDAR / RADx Data Hub Metadata Validator

bmir-radx / radx-project

This repo serves as a primary location for tracking issues that don't quite fit into our other dedicated repositories

0 stars 0 forks source link

Develop CEDAR / RADx Data Hub Metadata Validator #45

Open marcosmro opened 10 months ago

marcosmro commented 10 months ago

Develop a tool to check that metadata adheres to certain standards. This tool could be helpful as part of the RADx Data Hub submission workflow, at the validation step.

Examples: check that the JSON conforms to the template, check all the optional/recommended/required fields are filled out, compute a validation score, etc.

Part of https://github.com/bmir-radx/radx-project/issues/28

@marcosmro: Reviewer @matthewhorridge: Reviewer

yancao77 commented 9 months ago

The library hits the targeted minimum functionality, including:

Validating the template and instance files as valid JSON.
Validating the template is a valid CEDAR template.
Conducting JSON Schema Validation.
Verifying that the required fields are not empty.

yancao77 commented 8 months ago

Incorporate a data type validation component into the library. This component encompasses various validations tailored to specific field types, as outlined below:

Text Field: Conduct validation checks for length, regex, and literal values.
Numeric Field: Ensure the field values adhere to specified range constraints, data type, number of decimal places, and validate the '@type' attribute value.
Temporal Field: Validate the data type and the '@type' attribute value.
Link Field: Confirm that the input is a valid URL.
Controlled List: Initially validate to ensure that both '@id' and 'rdfs:label' entries exist and that their corresponding values are not null.

yancao77 commented 8 months ago

Add cardinality validation for both fields and elements into the library.
Add attribute-value fields validation.
Turn off all JSON schema validation warnings as per our team's consensus.

yancao77 commented 7 months ago

Develop validation specific to RADx to ensure the accuracy of three key RADx fields: Data File Identity/File Name, Data File Data Dictionary/Data Dictionary File Name, and Data File Identity/SHA256 digest.
Release version 1.0.0 on Maven Central

yancao77 commented 7 months ago

Recent updates have been made to guarantee the integration of the metadata validator:

version 1.0.1: Updated to replace snapshot dependencies.
version 1.0.2: Defined an alias for CarnidalityValidatorComponent bean to avoid conflicts with a bean of the same name from the Data Dictionary Validator.
version 1.0.3, Defined an alias for ValidatorReportWriter bean to avoid similar conflicts as mentioned above.

Version 1.0.3 has been successfully integrated into the Data Hub.