[Bug]: Facte mapping not working in current JSON mapping

JosefKaltwasser commented 3 months ago

What happened?

During our tests we stumbled over an issue with the current JSON mapping. The new JSON mapping is actually not an entity in GitHub yet. I could have allocated the to Methodology, but then I would talk to myself. Or I could have allocated it to tool, but I saw a risk for that to get lost in small tool issues. So, I raise it by email for now.

In the current JSON mapping we still use JSON inheritance for simple types. I invite a discussion for the TB whether we should get rid of that as well, as we did for classes, but of course we would open up some complexity when having to specify the semantics of cascades of facets manipulating lexical and/or values space along a path through the simple type inheritance tree, where all these theoretical considerations never occur in reality.

Nevertheless, what we do is inherit the types, and we simply map DATEX II facets to JSON Schema validation keywords. I summarise the situation taking the D2Datatypes “String” and “CountryCode” as an example:

"String": {
  "type": "string",
  "maxLength": 10
},
"CountryCode": {
  "$ref": "LevelC_3_Common.json#/definitions/String",
  "maxLength": 2
},

“String” is mapped to the the JSON type “string”, with a validation keyword maxLength of “10” (it is actually 1024 in our model, but wante to ease visualisation). “CountryCode” inherits from “String”, adding a more strict validation keyword maxLength of “2”. This does not work:

Obviously, the new validation keyword maxLength is not associated with the lexical space of the string. Me far from being a JSON expert, I asked ChatGPT to make this work and he came up with the following:

know that you are reluctant to use ‘complicated’ features like allOf in our JSON schema mapping, but then I find it still misleading to put the non-working maxLength into Country Code. So for me, the current mapping is not acceptable. We would have to: • either try our luck with the allof, or • resolve the inheritance – like we did for classes – and make CountryCode of type “string”, with application of “maxLength”: 2 (which is not a general solution as explained above, but would work for the moment, I believe), or • not map facets at all in our JSON mapping and allow all sorts of strings not compliant with out PIM.

Version

3

Code of Conduct

[X] I agree to follow this project's Code of Conduct

JosefKaltwasser commented 3 months ago

Mail from Jonas, 20/07/2024: Hi Josef, Yes, I think you found a bug there. When I did the new mapping I looked at the datatypes and first thought that I had to do changes but then I saw that we didn’t use inheritance with “advanced” features in JSON. We use a Reference. But as your tests shows it does not work with facets the way we do it. I think the solution is to do as we do with classes, we copy the type from the parent. CountryCode would then look like "CountryCode": { "type": "string", "maxLength": 2 },

jonas-jaderberg commented 2 months ago

I found out that this depends on what version of json schema you use. If I switch to something newer like

{ "$schema": "https://json-schema.org/draft/2019-09/schema", "description": "Comment describing your JSON Schema", "type": "object", "properties": { "Test": { "$ref": "#/definitions/CountryCode" } }, "definitions": { "String": { "type": "string", "maxLength": 10 }, "CountryCode": { "maxLength": 2, "$ref": "#/definitions/String" } }, "additionalproperties": false }

and an instance {
"Test":"testar333dd" }

It validates against the maxLength 2

I found this documented here https://json-schema.org/UnderstandingJSONSchema.pdf

Under 5.3 $ref

This means that if we have something newer than draf 7 our current solution works.

jonas-jaderberg commented 2 months ago

OpenAPI 3.1 support 2020-12, so wo should go for "$schema": "https://json-schema.org/draft/2020-12/schema",

paalaas commented 2 months ago

We can run our tests again for compliance when this is done in the tool.

DATEX-II-EU / DatexII