Guideline on validation of JSON-LD non-standard user extensions

[dakbhavesh]

This may be less of value from EPCIS 2.0 standard perspective however IMO it holds very high value for implementors in practice. As discussed in the last GSMP call, creating an issue here so that we can get opinions from group members and industry experts.

Question: In practice, how do we validate JSON-LD user extension fields syntactically and semantically?

Context/Example:

User extension fields are expressed through CURIE (Compact URI Expression). Refer example where example:myField is a user extension field. Each field has a definition documented in a specified namespace which is usually an RDF format.

It is quite possible that many industries would have ontology defined in terms of an RDF and have their own home-grown mechanism to validate business-specific data i.e. ILMD, user extensions, etc.

Industries would love to adopt EPCIS 2.0 therefore having comprehensive guidelines/approaches for validating standard and user extension fields would make adoption and decision making very easy.

cc: @nathanclevenger, @mgh128, @VladimirAlexiev, @shalika-singh, @RalphTro, @sboeckelmann

[VladimirAlexiev]

There are two aspects that qualify this question:

• JSONLD is both JSON and RDF, so do you want to validate at the level of JSON or RDF? (Or why not also XML)
• Do you want to validate simple structure (classes, props, cardinality, lookup lists), or also cross-field dependencies? (#255)

XML:

• structure: XSD (or RelaxNG)
• cross-field: Schematron

JSON:

• structure: JSON Schema
• cross-field: not sure, but I think there are various

RDF:

• structure: ontologies and SHACL
• cross-field: SHACL
• see https://github.com/validatingrdf/validatingrdf.github.io/wiki/Updated-list-of-implementations cc @shalikasingh

[sboeckelmann]

We (@dakbhavesh, @shalika-singh and me), were also discussing validation requirements and whether it should be specified in detail as part of the EPCIS 2.0 standards. It's pretty obvious that the capture components at least have to provide syntax validation, but the more I was looking into frameworks available for semantic validation, the more I came to the conclusion that they differ a lot in regard to supported functionality for various programming languages.

So I think we should separate the concerns and only require syntax validation. Nevertheless we will strive to fully support XSD, JSON Schema and SHACL based semantic and ontology validation as part of our implementation. But this really requires deep knowledge in validation framework requirements, so I would personally vote against putting that much pressure upon all of the implementors. Because a team that's developing a solution in nodejs might take a more JSON Schema oriented approach, while in other teams a XSD / Schematron approach might be more suitable, especially if xsd based validation can be derived from previous EPCIS 1.2 functionality. Introducing JSON-LD and SHACL is brilliant, but we should allow implementors to adopt from pure XML/XSD validation to JSON-LD at their own pace. That's why I think we should completely let the implementors decide to what extend they want to support semantic validation of user extension fields.

The list of implementations @VladimirAlexiev provided is really helpful and will surely allow us to support nice validation mechanisms within our java and javascript based tech-stacks. Thank you! I would really be looking forward to see an active ECPIS semantic web and linked data community for knowledge sharing. A community driven knowledge exchange and best-practices documentation would be something I'd prefer, rather than adding lots of requirements into the standard.

[VladimirAlexiev]

@mgh128: "Schematron has certainly existed longer than SHACL and is therefore likely to have a greater number of mature implementations, although I note that there are a number of open source implementations at github.com/search?q=shacl

See https://github.com/validatingrdf/validatingrdf.github.io/wiki/Updated-list-of-implementations. We'll take the effort to update this list from https://github.com/search?q=shacl and https://github.com/topics/shacl

As far as I'm aware, Schematron is primarily intended for validating XML data. I read ( schematron.com/2018/11/schematron-validation-of-json-data ) that a Schematron-inspired tool has been developed to validate JSON data.

Schematron is based on XPath. There is JsonPath, and "schematron json" shows promising examples, including https://github.com/amer-ali/jsontron

do you know if Schematron can validate Linked Data / JSON-LD ?

JsonTron or a similar tool can validate JSONLD as JSON. There certainly is a lot of value in "single sourcing" the cross-field validations, so that one source of definitions (with perhaps some minor adjustments to be done automatically) can be used for both XML and JSON validation.

Re RDF validation, graphs are fundamentally different from trees (XML, JSON), so I don't think Schematron is appropriate. Eg SHACL can/will check things beyond the JSONLD payload, at least that target nodes have appropriate types (see #206).

---

@sboeckelmann: I would really be looking forward to see an active ECPIS semantic web and linked data community for knowledge sharing

Do you know of other like-minded people? Where such community could be setup?

• Maybe at https://www.w3.org/community/groups/ ? It's very easy and quick
  • I searched for "transport" and for "logistics" and found only https://www.w3.org/community/autowebplatform/ which is not at all about logistics
• Or as some sort of GS1 group?

[CraigRe]

Closing as we are not providing schematron validation.