COVESA / vehicle_signal_specification

Vehicle Signal Specification - standardized way to describe automotive data
Mozilla Public License 2.0
322 stars 166 forks source link

JsonSchema Version of VSS #457

Open alexjoybc opened 2 years ago

alexjoybc commented 2 years ago

Hi this is a feature request

Adding jsonschema support for the specification

Why:

JsonSchema is machine readable and can be use for generating code, validating structure and is widely adopted by the community in general.

erikbosch commented 2 years ago

I assume no one would object to adding a jsonschema, could be a good addition. I assume it needs to be used after converting *.vspec files to json as the VSS include handling for branches is not standard json.

But as usual - it relies on that someone has the time and interest to do it. We would be more than happy if you would have the time to work on it and create a pull request.

SebastianSchildt commented 2 years ago

I tried looking into it in the past, but could not figure out (not a jsonschema expert), how to model the "meta" Aspects. It is relatively straightforward in jsonschema to say, "I want this JSON object to have an element foo which is required and string and an optional bar element that must be int", however for vspec we would need something along the lines of "every nested object needs to have a type", "If type equals sensor we can have...."

-> Not sure it is possible. If yuo know more about jsonschema @alexjoybc we could use some help/pointers here :)

alexjoybc commented 2 years ago

I would like to help for sure, we are evaluating using vss for one of our project and having jsonschema available would be a good starting point for us.

tguild commented 2 years ago

VSS' core truth is in YAML and both the current JSON and VSSo are derived through tooling. Those resulting formats have no immediate codependency so such a change should be fine. It is worth noting however that JSON-LD is more conducive towards ontologies while also retaining the benefits of JSON. On the other hand there seems to be more tooling, including code generators, and wider adoption of JSONSchema.

alexjoybc commented 2 years ago

Yes I agree, jsonschema should be generated through tooling without impacting the current JSON definition and not impact the yaml core definition. JsonSchema has greater adoption by the web community over JSON-LD, but again, following the guidelines of vss JsonSchema or JSON-LD should be supported over tooling, therefore one doesn't prevent the other and should be driven by community.

danielwilms commented 2 years ago

How to move forward:

alexjoybc commented 2 years ago

find bellow an example of using vss jsonschema with open api v3:

{
    "openapi": "3.0.1",
    "info": {
        "title": "VSS with open api",
        "description": "",
        "version": "1.0.0"
    },
    "paths": {
        "/vehicle": {
            "post": {
                "summary": "Add telematics data for a vehicle",
                "requestBody": {
                    "description": "Vehicle telematics data",
                    "content": {
                        "application/json": {
                            "schema": {
                                "$ref": "#/components/schemas/Vehicle"
                            }
                        }
                    },
                    "required": true
                },
                "responses": {
                    "200": {
                        "description": "Success",
                        "content": {}
                    }
                },
                "x-codegen-request-body-name": "vehicle"
            }
        }
    },
    "components": {
        "schemas": {
            "Vehicle": {
                "type": "object",
                "properties": {
                    "Vehicle":
                    {
                        "$ref": "./vss_rel_2.2.json#/Vehicle" 
                        "description": "this is a reference to the Vehicle object described in the jsonSchema, could also be like ./vss_rel_2.2.json#/Vehicle/ADAS"
                    }
                }
            }
        }
    }
}

Then the VSS specification in jsonSchema

I made the following transformations without much thinking, just to get it started ;-)

VSS data type JsonSchema data type
UInt8 integer
Int8 integer
UInt16 integer
Int16 integer
UInt32 integer
Int32 integer
UInt64 integer
Int64 integer
Boolean boolean
Float number
Double number
String string
{
    "Vehicle": {
        "properties": {
            "ADAS": {
                "properties": {
                    "ABS": {
                        "properties": {
                            "Error": {
                                "type": "boolean",
                                "description": "Indicates if ABS incurred an error condition. True = Error. False = No Error.",
                                "vss_type": "sensor",
                                "uuid": "cd2b0e86aa1f5021a9bb7f6bda1cbe0f"
                            },
                            "IsActive": {
                                "type": "boolean",
                                "description": "Indicates if ABS is enabled. True = Enabled. False = Disabled.",
                                "vss_type": "actuator",
                                "uuid": "433b7039199357178688197d6e264725"
                            },
                            "IsEngaged": {
                                "type": "boolean",
                                "description": "Indicates if ABS is currently regulating brake pressure. True = Engaged. False = Not Engaged.",
                                "vss_type": "sensor",
                                "uuid": "6dd21979a2225e31940dc2ece1aa9a04"
                            }
                        },
                        "description": "Antilock Braking System signals",
                        "vss_type": "branch",
                        "uuid": "219270ef27c4531f874bbda63743b330"
                    },
...
gunnarx commented 2 years ago

I am not convinced (yet) that converting VSS definitions to JSON schema really makes sense, or does it? To me (the VSS signal definition) and JSON schema, don't seem to serve the same purpose in a way (see further below). At least not without adding some more details to the intended outcome and what is actually desired here?

without impacting the current JSON definition

Please explain first what you mean by the current JSON definition, so we're not missing something.

EDIT: OK, I read the whole thread again and see now that @tguild mentioned the words 'current JSON'. I assume then that it is simply the VSS definitions as written in YAML, converted to JSON, using the existing vspec2json.py script. That's very much a one-to-one equivalence, with the addition of generated UUIDs, since YAML and JSON can be very closely equivalent. I guess I am saying that the conversion is in a sense obvious and a little surprised it is so often pops up in discussions because it seems to detract from core of the issue. From my perspective there's "little to talk about" regarding that very obvious one-to-one translation?

I made the following transformations without much thinking, just to get it started ;-)

Unless I am misunderstanding you I see something that I think is a common mistake/pattern here and that is to mix up the definition of signals with the definition of a value-transfer protocol or API for those signals. When you perform the POST operation defined in the API, and you send a document that follows the format described in the JSON schema, how do you transfer values of the signals, or is that not what you are trying to do? (Hint: there is no property named "value" in what you have described)

If that's not it, one alernative is that we want to check if new VSS definitions (that are written in YAML, that's the VSS way) follow the required format. (Here as @SebastianSchildt noted, that would require a little bit of additional logic, and without that logic it seems again, well, trivial. We have something like name, type, datatype and description and that's all.
So, as of now, the converter tools do the checking of the inputs, perhaps somewhat informally, and if it's going to be done more 'formally' then it seems that a YAML schema is rather required?

@alexjoybc I guess my rambling reaction to this thread is simply a plea that you explain the underlying intention and how it would be used. Essentially 'it's cool to convert to JSON schema' doesn't seem a useful reason to me. Well, sometimes it is ;-) but the output has to make some logical sense, so which of my speculative reasons mentioned here best matches the intention, (if any)?

The final alternative I see is that we really are aiming to define how to send a representation of the signal definitions themselves across some web API, as indicated by the web API description above. If we want to send the definitions over as a JSON document, which of course makes sense in typical web APIs, then couldn't the JSON format be 'defacto' defined by the aforementioned vspec2json.py converter? Along the same lines -> as far as APIs and definitions go, I thought the VISSv2 protocol had the ability to ask the server to return a list of signals, including their metadata (type etc)? I would link directly to the section but I failed at finding it in the specification just now (can @tguild help?) Why not look at the format returned there -- even if the whole VISSv2 was not used for some reason, it seems we could just as well stay compatible with the JSON format defined for that exchange.

Going back to my original feeling that JSON schema is best used to define the required format of the JSON document exchanged on a Web API (and not a definition of signals source document), I perhaps see the possibility to use JSON schema to more formally describe the format of JSON exchanges defined in the VISSv2 specification?

adobekan commented 1 year ago

Here is one more nice example on how to use jsonSchema. Might help with clarification. https://www.asyncapi.com/blog/json-schema-beyond-validation

gunnarx commented 1 year ago

Here is one more nice example on how to use jsonSchema. Might help with clarification.

That depends what you want to clarify and to whom. I know what JSON schema does - it describes the structure of a JSON document, and tools can check if a JSON document is correctly formatted, according to the schema.

The question in the discussion above is another: "What kind of document is it that you you want to check?"

adobekan commented 1 year ago

I know what JSON schema does - it describes the structure of a JSON document, and tools can check if a JSON document is correctly formatted, according to the schema.

Not that I was trying to refer to something in your comment. But I am happy for you that you know what for JSON schema is used. My comment was going more into the direction of one more general example of VSS and JSON Schema.

"What kind of document is it that you you want to check?"

At the end, if you use JSON to serialize data in this case VSS data, then you can use json-schema to validate your API req/response. Yes you are right it could be for ViSSv2 or any other API that is handling in this case VSS-data.

Cheers

timothygoodwill commented 1 year ago

Tying HTTP sync patterns to a VISSv2-like 'do anything' schema does not make the best use of OpenAPI, or HTTP semantics, let alone REST.

Taking a subdomain resource approach (multiple versioned API specification docs), with paths and JSON schema aligned to the VSS, and with filter and field-selection support would…

This approach is in-line with the VISSv2 vision

VISS2 Core example:

This approach would also open up an opportunity to model well-described AsyncAPI interfaces with a shared JSON schema for those use-cases that demand it.

tguild commented 1 year ago

@alexjoybc there is a pending PR for this in related repo

https://github.com/COVESA/vss-tools/pull/296

alanmapleburl-au commented 1 year ago

I'll add my two cents. I think it would be beneficial to have the source of truth in JsonSchema or OpenAPI with a tool to generate legacy vspec for backward compatibility. They're both more expressive than vspec and are well-known.