Open-Telecoms-Data / lib-cove-ofds

Other
0 stars 0 forks source link

Enums on spans are raising false positive alerts, jsons schema validate #81

Open odscjames opened 1 year ago

odscjames commented 1 year ago

The error occurs on the following fields:

odscjames commented 1 year ago

It's a schema definition problem. All 3 fields are actually arrays of items from code lists.

But looking at the schema, each of these fields has "codelist" and "openCodelist" on them at the top level of the field, where as they should have them at the lower level of the array "items".

When we used compiletojsonschema to make the schema, that tool put the enums in at the top level of the field, and then technically, the false positive error message we are seeing is correct.

['fibre'] is not one of ['fibre', 'microwave', 'copper', 'coaxial']

No it's not - the tool is expecting to see 'fibre' there ie a string not in a list.

See commit in Schema repo linked to this issue - when schema is recompiled with that it works. (And the commit in this repo adds a new test for this, as clearly we didn't pick this up before)

(All other Array of Enum fields should be checked - schema / technologies, for instance.)

There is an open question tho about how compiletojsonschema works with the schema - the schema files have both the "codelist"/"openCodelist" fields AND the actual Enum values. This means there is both a duplication of effort and the potential for these to get out of sync. I'm not sure what best way to fix up schema is here.

duncandewhurst commented 1 year ago

Why is compiletojsonschema being used? The Open Fibre schema doesn't require compilation.

Whilst I agree that it makes more sense for the codelist and openCodelist keywords to appear on items rather than directly on the property, those keywords are our own extensions to JSON Schema and the schema was written according to CoVEs behaviour at the time, which was to look for those keywords on the property.

When I replace the schema files in data with the uncompiled (but still dereferenced) versions from the standard repository, CoVE behaves as expected. Right now, I'd prefer to fix the tool rather than update the schema and release a new version of the standard just because the tool's behaviour changed.

odscjames commented 1 year ago

When I replace the schema files in data with the uncompiled (but still dereferenced) versions from the standard repository

Unless I'm getting wrong which files you copied, there's a "$ref": "https://raw.githubusercontent.com/Open-Telecoms-Data/open-fibre-data-standard/0__3__0/schema/network-schema.json" in one of the files which means when the tool is used it'll do a unneeded network call and lock in a GitHub dependency. We've had problems in libcove's before when they did external network calls. Also, I start getting test differences in additional fields tests.

I think I can see another solution, in changing how the compiletojsonschema tool works - it could detect arrays of strings and put enums in the right place.