tests: there is no check in place for schema validity.

[bkp7]

There are currently no automated tests in place for schema validity and these should be added to prevent recurrence of invalid schemas (issues #434 and #320).

The current json validator in use (TV4) does not check for schema validity as part of its tests.

[rob42]

Ive just tried the schema validator here https://github.com/java-json-tools/json-schema-validator and it seems quite capable. There is an online version at http://json-schema-validator.herokuapp.com/ but it assumes a one page schema, so that has minimal use for us.

I built a unit test that loads the schema from https://signalk.org/specification/1.0.0/schemas, which then reports detailed issues with a sample json. That could be wrapped into a web validator pinned to signalk versions, so samples can be tested online.

Output example:

16:53:00.217 [main] DEBUG nz.co.fortytwo.signalk.artemis.server.SignalkSchemaTest - error: JSON value is of type array, not a JSON Schema (expected an object)
    level: "error"
    schema: {"loadingURI":"https://signalk.org/specification/1.0.0/schemas/groups/electrical.json#","pointer":"/definitions/chargerQualities/properties/allOf"}
    domain: "syntax"
    found: "array"

16:53:00.217 [main] DEBUG nz.co.fortytwo.signalk.artemis.server.SignalkSchemaTest - Validating sample:docs-data_model_metadata.json
16:53:00.478 [main] DEBUG nz.co.fortytwo.signalk.artemis.server.SignalkSchemaTest -     Is valid? docs-data_model_metadata.json : true
16:53:00.758 [main] DEBUG nz.co.fortytwo.signalk.artemis.server.SignalkSchemaTest - Validating sample:vessel-time.json
16:53:00.778 [main] DEBUG nz.co.fortytwo.signalk.artemis.server.SignalkSchemaTest -     Is valid? vessel-time.json : false
16:53:00.779 [main] DEBUG nz.co.fortytwo.signalk.artemis.server.SignalkSchemaTest - error: string "2014-04-10T08:33:530Z" is invalid against requested date format(s) [yyyy-MM-dd'T'HH:mm:ssZ, yyyy-MM-dd'T'HH:mm:ss.[0-9]{1,12}Z]
    level: "error"
    schema: {"loadingURI":"https://signalk.org/specification/1.0.0/schemas/definitions.json#","pointer":"/definitions/timestamp"}
    instance: {"pointer":"/vessels/urn:mrn:signalk:uuid:b7590868-1d62-47d9-989c-32321b349fb9/environment/time/timestamp"}
    domain: "validation"
    keyword: "format"
    attribute: "date-time"
    value: "2014-04-10T08:33:530Z"
    expected: ["yyyy-MM-dd'T'HH:mm:ssZ","yyyy-MM-dd'T'HH:mm:ss.[0-9]{1,12}Z"]

[rob42]

Needs http access to the different versions of the schema (v1.0.0 differs from v1.0.4 ), currently I only see v1.0.0. I'm very short on time, so I dont want to start on this if there is another solution underway, should I go ahead?

[webmasterkai]

https://ajv.js.org might be a useful tool in validating.

[bkp7]

@rob42, sorry perhaps my post was misleading. There is no problem in getting the existing TV4 validator to validate the schema, it just that such a test hasn't been implemented yet.

All other json validation tools are useless to us at present as none support the TV4 'banUnknownProperties' flag which our schemas require in order to work properly (see issue #246).

We already have an online validator tool at http://node-master.signalk.org/@signalk/playground/dist/ which uses TV4.

The current proposal is to update the existing schema to be draft-04 compliant thus removing the dependency on TV4. At that point any json validator can be used for validation and the proposal is to move to AJV. This will also give us coverage stats for our tests.

[rob42]

ah - ok. I'll leave it then