Modular JSON Schema. Usage of bare terms allowed. Validation of extensions.

[VladimirAlexiev]

@jmcanterafonseca-iota: We have separate folders for the schemas and examples using different technologies:

• Ontology (and SHACL shape) vs Turtle
• XSD vs XML (@sboeckelmann agreed to this and I think has already done it in his XML-related PRs)

I suggest to do the same for JSON:

• move JSON/modular-schema to JSON-schema Once your modular schema is adopted, it will be the EPCIS JSON schema, so there's no need to keep the word "modular"
  • the old schemas EPCIS-JSON-Schema.json EPCIS-JSON-Schema-single-event.json EPCISMasterData-JSON-Schema.json can be moved to JSON-schema/old/ or removed altogether
  • this will leave folder JSON/ only for examples JSON/**.jsonld (we already have multiple subfolders of JSON examples)
• move JSON/modular-schema/test/*.json to JSON/modular-schema/*.jsonld (i.e. "examples testing the modular schema"), so the same examples can also be converted to Turtle and diagrammed.

As per https://github.com/gs1/EPCIS/issues/277#issuecomment-863030598, will you validate jsonld examples against the JSON schemas? Is that what validate.sh does, or it checks the referential integrity of the schema?

Cheers!

[VladimirAlexiev]

@jmcanterafonseca-iota thanks!

• enumerated fields allow extension URLs as value (good!):
  • "anyOf": [{"$ref": "definitions-JSON-Schema.json#/definitions/vocab-uri"}, {"enum": [...]
• "vocab-uri": "format": "uri" disallows CURIEs but it's very likely extensions will want to use CURIEs (eg rail:...)
• On real deployment, the base URL of all schemas should be changed from https://gs1.github.io/EPCIS/JSON-Schema/schemas to something more appropriate (similar to context and ontology). I think we don't have a task for deploying the schemas, maybe post one?
• "component" is merely "$ref": "#/definitions/string" but should be changed to an enum, see #264.
  • Base cbv:COMP-
  • Enum values X, Y, Z, Azimuth, AxialDistance, SphericalRadius, PolarAngle, Latitude, Longitude, Elevation, Easting, Northing, GeoCoordinates
• Please move JSON-Schema/test/ to JSON/schema-test/ so they're together with all other JSON examples
• Please change the validation script to check all examples not just the specially made tests

[VladimirAlexiev]

Answers at the telco today:

• "format": "uri" does allow CURIEs (demonstrated at jsonvalidate.com or some similar site)
• JSON-Schema/test/* includes single-event tests. If approved, it will be moved to JSON/single-event/*
• enums (to be added both in JSON-LD-context and JSON-schema):
  • "component" enum will be added
  • @jmcanterafonseca-iota do you have gs1:MT- ?
  • @mgh128 and @RalphTro are we forgetting any other enum?

[VladimirAlexiev]

@jmcanterafonseca-iota A creative rascal can go around your restriction "custom URLs should be outside the CBV namespace" in various simple ways, eg:

• prefix cbv1: expanding to https://ns.gs1.org/cbv/ then cbv1:Disp-active
• prefix mydisp: expanding to https://ns.gs1.org/cbv/Disp- then mydisp:active
• etc etc

[shalikasingh]

Hi @jmcanterafonseca-iota,

As discussed during the last call, we believe that we are planning the change to the format of request payload for POST /eventTypes/{eventType}/events endpoint. (i.e., from capturing single event within an EPCIS document to capturing a single event as a raw element without being wrapped in any other object(such as epcisBody)) then apart from JSON schema below are places that will entail change for the same

1. openapi.yaml : endpoint POST /eventTypes/{eventType}/events
2. hash-generator service
3. remove JSON/EPCIS-JSON-Schema-single-event.json

It would be nice to take care of the above changes as part of your PR only.

[shalikasingh]

Hi @jmcanterafonseca-iota, @mgh128, @CraigRe

Would like to bring into notice that the example in the EPCIS repo is invalidating against the JSON schema proposed.

We should either adjust JSON schema or example.

Below has been quoted from Section 7.4.2 Object Event of 2021_06_14l EPCIS 2-0 COMMREV

An ObjectEvent SHALL contain either a non-empty epcList , a non-empty quantityList , or both. The only permissible exception is if the object of observation is a physical location – in this case, the ObjectEvent SHALL contain a sensorElement and a non-empty readPoint populated with a physical location ID.

Which makes us believe that JSON schema should be adjusted here but an expert opinion will help.

[CraigRe]

@mgh128 @CraigRe @jmcanterafonseca-iota JSON Schema & SHACL need to be tweaked to reflect Shalika's comment:

An ObjectEvent SHALL contain either a non-empty epcList , a non-empty quantityList , or both. The only permissible exception is if the object of observation is a physical location – in this case, the ObjectEvent SHALL contain a sensorElement and a non-empty readPoint populated with a physical location ID.

[VladimirAlexiev]

But @shalikasingh what is the error?

I think I see one point where the example needs to be adjusted: remove gs1:MT- and use pure string.

[CraigRe]

This PR needs a final decision; what will we allow apart from bare terms (i.e., CURIEs, full URIs)? To re-visit on 24 August call.

[VladimirAlexiev]

what will we allow apart from bare terms (i.e., CURIEs, full URIs)?

@CraigRe We have to allow both. As @jmcanterafonseca-iota explained, "format": "uri" does allow URIs and CURIEs

[shalikasingh]

Hi @visibleOrigins, @mgh128, @jmcanterafonseca-iota, @CraigRe, @VladimirAlexiev,

We found a critical bug in modular JSON schema which we feel should be given attention as soon as possible,

Consider an EPCIS Document like below:

{
    "@context": [
        "https://gs1.github.io/EPCIS/epcis-context.jsonld",
        {
            "example": "http://ns.example.com/epcis/"
        }
    ],
    "id": "_:document1",
    "isA": "EPCISDocument",
    "schemaVersion": "2.0",
    "creationDate": "2019-11-01T14:00:00.000+01:00",
    "epcisBody": {
        "eventList": [
            {
                "eventID": "ni:///sha-256;025ac144187a8c5e14caf4d1cfa69250a33dc59a5bc42a68d31b1b5e55a3f15a?ver=CBV2.0",
                "isA": "AssociationEvent",
                "eventTime": "2019-11-01T14:00:00.000+01:00",
                "eventTimeZoneOffset": "+01:00",
                "parentID": "urn:epc:id:grai:4012345.55555.987",
                "childEPCs": [
                    "urn:epc:id:giai:4000001.12345"
                ],
                "action": "ADD",
                "bizStep": "assemble",
                "readPoint": {
                    "id": "urn:epc:id:sgln:4012345.00001.0"
                }
            }
        ]
    }
}

It has only one problem: the attribute "bizStep" value is not as expected.(Bare string is only allowed for CBV defined by GS1). But the current schema produces below error message:

<ProblemResponseBody>
    <type>epcisExceptions:CaptureValidationException</type>
    <title>Validation failed</title>
    <status>400</status>
    <detail>1) #: required key [eventTime] not found 2) #: required key [eventTimeZoneOffset] not found 3) #: required key [action] not found 4) #/isA: EPCISDocument is not a valid enum value 5) #: required key [epcList] not found 6) #: required key [quantityList] not found 7) #: required key [sensorElementList] not found 8) #: required key [readPoint] not found 9) #: required key [eventTime] not found 10) #: required key [eventTimeZoneOffset] not found 11) #: required key [action] not found 12) #/isA: EPCISDocument is not a valid enum value 13) #: required key [eventTime] not found 14) #: required key [eventTimeZoneOffset] not found 15) #: required key [action] not found 16) #: required key [parentID] not found 17) #/isA: EPCISDocument is not a valid enum value 18) #: required key [eventTime] not found 19) #: required key [eventTimeZoneOffset] not found 20) #/schemaVersion: schemaVersion is not a valid enum value 21) #/schemaVersion: [schemaVersion] is not a valid URI 22) #/schemaVersion: schemaVersion is not a valid enum value 23) #/epcisBody: epcisBody is not a valid enum value 24) #/epcisBody: [epcisBody] is not a valid URI 25) #/epcisBody: epcisBody is not a valid enum value 26) #/id: id is not a valid enum value 27) #/id: [id] is not a valid URI 28) #/id: id is not a valid enum value 29) #/creationDate: creationDate is not a valid enum value 30) #/creationDate: [creationDate] is not a valid URI 31) #/creationDate: creationDate is not a valid enum value 32) #/isA: EPCISDocument is not a valid enum value 33) #: required key [inputEPCList] not found 34) #: required key [inputQuantityList] not found 35) #: required key [outputEPCList] not found 36) #: required key [outputQuantityList] not found 37) #: required key [transformationID] not found 38) #: required key [inputEPCList] not found 39) #: required key [inputQuantityList] not found 40) #: required key [outputEPCList] not found 41) #: required key [outputQuantityList] not found 42) #: required key [eventTime] not found 43) #: required key [eventTimeZoneOffset] not found 44) #: required key [bizTransactionList] not found 45) #: required key [action] not found 46) #/isA: string [EPCISDocument] does not match pattern ^TransactionEvent$ 47) #: required key [eventTime] not found 48) #: required key [eventTimeZoneOffset] not found 49) #/isA: [EPCISDocument] is not a valid URI 50) #/epcisBody/eventList/0/bizStep: assemble is not a valid enum value 51) #/epcisBody/eventList/0/bizStep: [assemble] is not a valid URI 52) #/epcisBody/eventList/0/isA: AssociationEvent is not a valid enum value 53) #/epcisBody/eventList/0: required key [epcList] not found 54) #/epcisBody/eventList/0: required key [quantityList] not found 55) #/epcisBody/eventList/0: required key [sensorElementList] not found 56) #/epcisBody/eventList/0/bizStep: assemble is not a valid enum value 57) #/epcisBody/eventList/0/bizStep: [assemble] is not a valid URI 58) #/epcisBody/eventList/0/isA: AssociationEvent is not a valid enum value 59) #/epcisBody/eventList/0/bizStep: assemble is not a valid enum value 60) #/epcisBody/eventList/0/bizStep: [assemble] is not a valid URI 61) #/epcisBody/eventList/0/action: action is not a valid enum value 62) #/epcisBody/eventList/0/action: [action] is not a valid URI 63) #/epcisBody/eventList/0/action: action is not a valid enum value 64) #/epcisBody/eventList/0/childEPCs: childEPCs is not a valid enum value 65) #/epcisBody/eventList/0/childEPCs: [childEPCs] is not a valid URI 66) #/epcisBody/eventList/0/childEPCs: childEPCs is not a valid enum value 67) #/epcisBody/eventList/0/parentID: parentID is not a valid enum value 68) #/epcisBody/eventList/0/parentID: [parentID] is not a valid URI 69) #/epcisBody/eventList/0/parentID: parentID is not a valid enum value 70) #/epcisBody/eventList/0/bizStep: assemble is not a valid enum value 71) #/epcisBody/eventList/0/bizStep: [assemble] is not a valid URI 72) #/epcisBody/eventList/0/isA: AssociationEvent is not a valid enum value 73) #/epcisBody/eventList/0: required key [inputEPCList] not found 74) #/epcisBody/eventList/0: required key [inputQuantityList] not found 75) #/epcisBody/eventList/0: required key [outputEPCList] not found 76) #/epcisBody/eventList/0: required key [outputQuantityList] not found 77) #/epcisBody/eventList/0: required key [transformationID] not found 78) #/epcisBody/eventList/0: required key [inputEPCList] not found 79) #/epcisBody/eventList/0: required key [inputQuantityList] not found 80) #/epcisBody/eventList/0: required key [outputEPCList] not found 81) #/epcisBody/eventList/0: required key [outputQuantityList] not found 82) #/epcisBody/eventList/0: required key [bizTransactionList] not found 83) #/epcisBody/eventList/0/isA: string [AssociationEvent] does not match pattern ^TransactionEvent$ 84) #/epcisBody/eventList/0/bizStep: assemble is not a valid enum value 85) #/epcisBody/eventList/0/bizStep: [assemble] is not a valid URI 86) #/epcisBody/eventList/0: required key [epcList] not found 87) #/epcisBody/eventList/0: required key [quantityList] not found 88) #/epcisBody/eventList/0/action: string [ADD] does not match pattern ^DELETE$ 89) #/epcisBody/eventList/0/isA: [AssociationEvent] is not a valid URI 90) #/epcisBody: required key [queryResults] not found 91) #/isA:  </detail>
    <instance/>
</ProblemResponseBody>

Whereas the expected error message should be as follows:

<ProblemResponseBody>
    <type>epcisExceptions:CaptureValidationException</type>
    <title>Validation failed</title>
    <status>400</status>
    <detail>1) #/epcisBody/eventList/0/bizStep: assemble is not a valid enum value 2) #/epcisBody/eventList/0/bizStep: [assemble] is not a valid URI </detail>
    <instance/>
</ProblemResponseBody>

We have attached the JSON schema below that fixes the above problem. @jmcanterafonseca-iota, you can either replace the schema with the one attached or if you wish we can push changes in your branch. Let us know your preference

modular-json-schema.txt

Note: The attached JSON schema is built on top of the latest modular schema in this PR (i.e proposed by @jmcanterafonseca-iota)

[dakbhavesh]

Hi @jmcanterafonseca-iota, Can you please make me a collaborator so that I can push changes proposed by @shalikasingh in the previous comment?

[dakbhavesh]

Hi @jmcanterafonseca-iota, I have pushed changes to consider isA property for accurate validation of events. Please take a look and let me know in case of any issues.

[VladimirAlexiev]

@jmcanterafonseca-iota, @dakbhavesh : Other problems found in examples made with OpenEpcis.io convertor (https://github.com/gs1/EPCIS/issues/276):

@RalphTro wrote "I do not understand why the JSON schema validation throws errors:

• (reason and correctiveEventIDs are not required in an errorDeclarationElement)
• “JSON does not match all schemas from “allOf”
• etc.
• so I think we need to double-check that the schema is correctly defined."