Implement schema-based validation for USDM JSON data

[ASL-rmarshall]

The DDF team has identified some generic checking requirements which might be best implemented through the checking of USDM JSON data against a defined USDM schema (e.g., JSON-Schema). These checks include confirming conformance with characteristics defined in the USDM API specification, including such things as :

• Primitive data types
• Correct class usage
• Cardinalities
• Required properties

[ASL-rmarshall]

This was the proposal I made at the beginning of April: USDM Schema Validation.pptx

Given that schemas in API specs based on OpenAPI v3.1 or higher are JSON-Schema compliant (and the USDM v3.0 API spec is OpenAPI v3.1), I have shown that it's possible to extract a usable USDM JSON-Schema definition directly from the USDM API spec, pass it into the JSON-Schema validator currently used by the CORE Engine to validate Dataset-JSON files (jsonschema), and use it to generate a list of schema violations found in a USDM JSON data file (e.g., CDISC_Pilot_Study_sv_edit-2024-05-16T13-19-27.xlsx).

The following would be needed from the CORE team to implement this proposal:

• Define a new Rule Type (e.g., "JSON Schema Check") that would trigger:
  • Retrieval of the appropriate schema (probably based on the specified standard and standard version). We'd need to figure out the best way to provision USDM schemas:
    • Is a separate specification needed or will the API spec alone be sufficient?
    • Access route: e.g., direct from GitHub, direct from CDISC Library, from GitHub via CDISC Library, etc.?
  • Execution of the JSON-Schema validation and gathering of validation errors (which should probably be cached so that repeated validation is not needed)
  • Comparison of rule specifications against validation errors and reporting of rule success/failure. We'll need to determine:
    • How the comparison is made. The jsonschema validator creates a hierarchical tree of validation errors, which can be interpreted in different ways (e.g., should an error found when comparing against one of the "subschemas" of an anyOf be reported individually or should there be a single error for the anyOf: 'xxx' is not valid under any of the given schemas).
    • The specificity of rules (e.g., one rule per schema component, per class, per attribute, etc.)
    • What information needs to be presented with each error message to make it interpretable.
• In order to develop and test schema-based rules, the Rules Editor would need to be updated:
  • To not report the new Rule Type as an error (schema update)
  • Allow JSON files containing test data to be loaded.
• Any new operators needed to assess schema validation errors. However, this proposal primarily involves defining rules according to schema validation error property values (e.g., message contains "xxx", validator equals "yyy"), so a need for new operators is not anticipated.

[ASL-rmarshall]

This is a table of the JSON-Schema attributes that are handled by the jsonschema validator:	Schema Attribute	Message(s)
$dynamicRef
$ref
additionalItems	Additional items are not allowed (%s %s unexpected)
additionalProperties	{joined} {verb} not match any of the regexes: {patternProperties} Additional properties are not allowed (%s %s unexpected)
allOf
anyOf	{instance!r} is not valid under any of the given schemas
const	{const!r} was expected
contains	{instance!r} does not contain items matching the given schema
dependentRequired	{each!r} is a dependency of {property!r}
dependentSchemas
enum	{instance!r} is not one of {enums!r}
exclusiveMaximum	{instance!r} is greater than or equal to the maximum of {maximum!r}
exclusiveMinimum	{instance!r} is less than or equal to the minimum of {minimum!r}
format
if
items	Expected at most {prefix} items, but found {total}
maxItems	{instance!r} is too long
maxLength	{instance!r} is too long
maxProperties	{instance!r} has too many properties
maximum	{instance!r} is greater than the maximum of {maximum!r}
minItems	{instance!r} is too short
minLength	{instance!r} is too short
minProperties	{instance!r} does not have enough properties
minimum	{instance!r} is less than the minimum of {minimum!r}
multipleOf	{instance!r} is not a multiple of {dB}
not	{instance!r} should not be valid under {not_schema!r}
oneOf	{instance!r} is not valid under any of the given schemas {instance!r} is valid under each of {reprs}
pattern	{instance!r} does not match {patrn!r}
patternProperties
prefixItems
properties
propertyNames
required	{property!r} is a required property
type	{instance!r} is not of type {reprs}
unevaluatedItems	Unevaluated items are not allowed (%s %s unexpected)
unevaluatedProperties	Unevaluated properties are not allowed (%s %s unexpected) Unevaluated properties are not valid under the given schema (%s %s unevaluated and invalid)
uniqueItems	{instance!r} has non-unique elements