More error information about failed validations

[anandp504]

I used the https://github.com/labra/shexjava project to check if the error gives more information about the element in the JSONLD data which failed a particular validation. I changed the "name" element in the wot.jsonld to an integer and ran the following command.

java -jar target/shexjava.jar -d examples/wot.jsonld -df JSON-LD -s examples/wot.shex -m examples/wot.shapeMap

The response from the validation is given below. Although it gives information about what the error is, we do not have information about the element/field in the JSONLD which actually failed the validation. I also noticed that the es.weso.schema.Result itself is valid in such cases and the ResultShapeMap status is coming out as "nonconformant". What will be the scenario in which Result.isValid function return false? We would like to have both the Result to be invalid and have more information about the elements which failed validation.

[
  {
    "node" : "<http://example.org/mySensor>",
    "shape" : "<Thing>",
    "status" : "nonconformant",
    "appInfo" : "Shaclex",
    "reason" : "Error: Attempt: node: <http://example.org/mySensor>, shape: <Thing>\n1234 does not have datatype xsd:string"
  }
]

[labra]

Indeed, the information about failed validations must be improved. At least the "reason" field could be changed to be an array of reasons.

Originally, the Result object was returning false when there was at least one node that didn't match a shape, but we later changed it and used the status field.

At this moment, isValid field would return false if there is a failure during the validation. It is true that the name can be misinterpreted, so I think it may be better to deprecate it in future versions of the validator.

[anandp504]

@labra Thank you for the response. Right now if we have a CLOSED shape and I include a field which is not present in the shex file, we do get back the field name in the error message. For e.g. Error: Closed shape. But rest teacher:testBirthDate is not in ^<http://www.w3.org/ns/shacl#targetNode> or However, if we have a mandatory field and the JSONLD or Turtle field does not have the mandatory field, then we get back a huge string with the error message. The following part from the error message has a subtle message which indicates the node C5 is missing. I will have to then extract which node corresponds to the node C5 and then get the field name, which looks very cumbersome. Bag {| C0 -> 1, C1 -> 1, C2 -> 1, C3 -> 1, C4 -> 1, C6 -> 1, C7 -> 1, C8 -> 1, C9 -> 1, C10 -> 1, C11 -> 1, C12 -> 1, C13 -> 1, C14 -> 1, C15 -> 1 |} does not match Rbe C0,C1,C2?,C3?,C4,C5,C6,C7,C8,C9,C10,C11,C12,C13,C14,C15

Please let me know if the error format will be changed sometime soon. I can give it a shot for changing the error format and give a PR.

[anandp504]

@labra The following are some of the various types of error messages we get from the shaclex library.

Error: Node 1990-12-06 has no shape teacher:isAString\nErrors: Error: Attempt: node: 1990-12-06, shape: teacher:isAString\n1990-12-06 does not have datatype xsd:string

Node Id: _:9c7454646d83e3bf8c35ba7b591e7e64 is NON-CONFORMANT with reason: Error: teacher:GenderTypeCode-INVALID does not belong to [<http://example.com/voc/teacher/1.0.0/GenderTypeCode-MALE>,<http://example.com/voc/teacher/1.0.0/GenderTypeCode-FEMALE>]

"Error: Node abc has no shape teacher:isADecimal\nErrors: Error: Attempt: node: abc, shape: teacher:isADecimal\nabc does not have datatype xsd:decimal\nDetails: Lexical form 'abc' is not a legal instance of Datatype[http://www.w3.org/2001/XMLSchema#decimal -> class java.math.BigDecimal] Lexical form 'abc' is not a legal instance of Datatype[http://www.w3.org/2001/XMLSchema#decimal -> class java.math.BigDecimal] during parse -org.apache.xerces.impl.dv.InvalidDatatypeValueException: cvc-datatype-valid.1.2.1: 'abc' is not a valid value for 'decimal'."

"Error: Node _:554f70048d62be60c0af24b9d3867e5e has no shape teacher:InServiceTeacherTrainingShape\nErrors: Error: Node abc has no shape teacher:isADecimal\nErrors: Error: Attempt: node: abc, shape: teacher:isADecimal\nabc does not have datatype xsd:decimal\nDetails: Lexical form 'abc' is not a legal instance of Datatype[http://www.w3.org/2001/XMLSchema#decimal -> class java.math.BigDecimal] Lexical form 'abc' is not a legal instance of Datatype[http://www.w3.org/2001/XMLSchema#decimal -> class java.math.BigDecimal] during parse -org.apache.xerces.impl.dv.InvalidDatatypeValueException: cvc-datatype-valid.1.2.1: 'abc' is not a valid value for 'decimal'."```

[labra]

Thanks, indeed, those messages are not very practical.

That part of the library is very unstable, specially if you are using the SHACL engine.

Our propose is to enrich the error information through the appInfo field which will contain a Json structure representing the errors and also to generate a Validation report from that structure which will comply with the spec.

We created issue #60 to describe the process of this conversion.

I will also keep this issue open until we have improved the error messages.

[steotia]

I tried to run the following to generate the report via showValidationReport

sbt:shaclex> run -e ShEx -s examples/shex/good1.shex --schemaFormat ShExC -d examples/shex/good1.ttl -v TURTLE --showValidationReport
[info] Running (fork) es.weso.shaclex.Main -e ShEx -s examples/shex/good1.shex --schemaFormat ShExC -d examples/shex/good1.ttl -v TURTLE --showValidationReport
[info] Result :x@:S
[info] Error: No validaton report in ShEx
[success] Total time: 4 s, completed 16 Apr, 2018 4:40:35 PM

Shex errored out.

sbt:shaclex> run -d examples/shacl/good1.ttl --engine ShaClex  -v TURTLE --showValidationReport
[info] Running (fork) es.weso.shaclex.Main -d examples/shacl/good1.ttl --engine ShaClex -v TURTLE --showValidationReport
[info] Result :x@:S,:x@137f5c61127599efbef10e70a6870c12,:y@:S,:y@137f5c61127599efbef10e70a6870c12
[info] @prefix :      <http://example.org/> .
[info] @prefix sh:    <http://www.w3.org/ns/shacl#> .
[info] [ a            sh:ValidationReport ;
[info]   sh:conforms  true
[info] ] .
[success] Total time: 4 s, completed 16 Apr, 2018 4:40:46 PM
sbt:shaclex>

ShaClex did not.

Is Shex not supported? Between Shex and ShaClex, which one is more stable and recommended?

[labra]

Both ShEx and SHACL are equally supported by SHACLex.

The main problem is that they are different technologies which had been developed by different means and with different philosophies. You can read more about the differences between ShEx and SHACL in chapter 7 of the Validating RDF Data book.

ShEx generates a shapeMap as the result of validation, that's why, in case of no errors, it returns :x@:S while SHACL generates also a shapeMap which can also be converted to a SHACL validation report.

In the case of ShEx, the conversion to the SHACL validation report could be possible but we haven't implemented it yet.