search

AKSW / RDFUnit

An RDF Unit Testing Suite
http://RDFUnit.aksw.org
Apache License 2.0
150 stars 42 forks source link

RDFUnit extendedResults are not ordered #35

Closed AndreAga closed 9 years ago

AndreAga commented 9 years ago

Hi, after some tests, I notice that if rdfunit doesn't find errors the output is like that:

@Prefixes....

rutr:4fb97fee-2acd-11b2-8041-aa98dd5aae63
        a                          rut:TestExecution , prov:Activity ;
        rut:source                 </Users/AndreAga/Documents/Sviluppo/Progetti/UnifiedViews/Datasets/Elezioni.ttl> ;
        rut:testsError             "0"^^xsd:nonNegativeInteger ;
        rut:testsFailed            "0"^^xsd:nonNegativeInteger ;
        rut:testsRun               "1594"^^xsd:nonNegativeInteger ;
        rut:testsSuceedded         "1594"^^xsd:nonNegativeInteger ;
        rut:testsTimeout           "0"^^xsd:nonNegativeInteger ;
        rut:totalIndividualErrors  "0"^^xsd:nonNegativeInteger ;
        prov:endedAtTime           "2015-03-25T21:24:26.962Z"^^xsd:dateTime ;
        prov:startedAtTime         "2015-03-25T21:24:24.399Z"^^xsd:dateTime ;
        prov:used                  ruts:4fb97fef-2acd-11b2-8041-aa98dd5aae63 ;
        prov:wasAssociatedWith     schema: , <http://xmlns.com/foaf/0.1/> ;
        prov:wasStartedBy          <http://localhost/> .

ruts:4fb97fef-2acd-11b2-8041-aa98dd5aae63
        a               rut:TestSuite , prov:Collection ; prov:hadMember .....

But if the tool finds errors the output is like that:

@Prefixes...

<http://rdfunit.aksw.org/data/results#861d2bf8-2acd-11b2-80c0-aa98dd5aae63/861d62bc-2acd-11b2-80c0-aa98dd5aae63>
        a                    rut:TestCaseResult , rut:RLOGTestCaseResult , rlog:Entry , rut:ExtendedTestCaseResult , spin:ConstraintViolation ;
        rlog:level           rlog:WARN ;
        rlog:message         "http://schema.org/address is missing proper range" ;
        rlog:resource        <Via Marino Mazzacurati, 90@it> ;
        dcterms:date         "2015-03-26T22:45:15.951Z"^^xsd:dateTime ;
        rut:testCase         rutt:schema-RDFSRANGE-MISS-3f67d88cc38f44bb98e89a21653c7dc0f5227586b7b712f54cfdf78761762f2a ;
        spin:violationPath   schema:address ;
        spin:violationRoot   <Via Marino Mazzacurati, 90@it> ;
        prov:wasGeneratedBy  rutr:861d2bf8-2acd-11b2-80c0-aa98dd5aae63 .

rutr:861d2bf8-2acd-11b2-80c0-aa98dd5aae63
        a                          prov:Activity , rut:TestExecution ;
        rut:source                 </Users/AndreAga/Documents/Sviluppo/Progetti/UnifiedViews/Plugins-QualityAssessment/Q-RDFUnit/target/test-classes/Scuole.ttl> ;
        rut:testsError             "0"^^xsd:nonNegativeInteger ;
        rut:testsFailed            "3"^^xsd:nonNegativeInteger ;
        rut:testsRun               "1594"^^xsd:nonNegativeInteger ;
        rut:testsSuceedded         "1591"^^xsd:nonNegativeInteger ;
        rut:testsTimeout           "0"^^xsd:nonNegativeInteger ;
        rut:totalIndividualErrors  "1232"^^xsd:nonNegativeInteger ;
        prov:endedAtTime           "2015-03-26T22:45:16.822Z"^^xsd:dateTime ;
        prov:startedAtTime         "2015-03-26T22:45:14.576Z"^^xsd:dateTime ;
        prov:used                  ruts:861d634b-2acd-11b2-80c0-aa98dd5aae63 ;
        prov:wasAssociatedWith     <http://xmlns.com/foaf/0.1/> , schema: ;
        prov:wasStartedBy          <http://localhost/> .

<http://rdfunit.aksw.org/data/results#861d2bf8-2acd-11b2-80c0-aa98dd5aae63/861d5bea-2acd-11b2-80c0-aa98dd5aae63>
        a                    rut:ExtendedTestCaseResult , rut:RLOGTestCaseResult , rlog:Entry , spin:ConstraintViolation , rut:TestCaseResult ;
        rlog:level           rlog:WARN ;
        rlog:message         "http://schema.org/address is missing proper range" ;
        rlog:resource        <Piazza Di S. Alessio, 34@it> ;
        dcterms:date         "2015-03-26T22:45:15.95Z"^^xsd:dateTime ;
        rut:testCase         rutt:schema-RDFSRANGE-MISS-3f67d88cc38f44bb98e89a21653c7dc0f5227586b7b712f54cfdf78761762f2a ;
        spin:violationPath   schema:address ;
        spin:violationRoot   <Piazza Di S. Alessio, 34@it> ;
        prov:wasGeneratedBy  rutr:861d2bf8-2acd-11b2-80c0-aa98dd5aae63 .

Is there a way to put the "summary" on the top of the file? after the prefixes?

jimkont commented 9 years ago

Hi Andrea,

This happens because RDF does not guarantee ordering and when we save as TURTLE any resource can be anywhere in the file If you want to read the overview results you can use the DatasetOverviewResults class

an example on how this is used can be found in https://github.com/AKSW/RDFUnit/blob/master/rdfunit-validate/src/test/java/org/aksw/rdfunit/validate/integration/PatternsGeneratorsIntegrationTest.java#L129-165

let me know if it helps

Dimitris

On Fri, Mar 27, 2015 at 1:06 AM, Andrea Agazzone notifications@github.com wrote:

Hi, after some tests, I noticed that if rdfunit doens't find errors the

output is like that:

@Prefixes....

rutr:4fb97fee-2acd-11b2-8041-aa98dd5aae63 a rut:TestExecution , prov:Activity ; rut:source ; rut:testsError "0"^^xsd:nonNegativeInteger ; rut:testsFailed "0"^^xsd:nonNegativeInteger ; rut:testsRun "1594"^^xsd:nonNegativeInteger ; rut:testsSuceedded "1594"^^xsd:nonNegativeInteger ; rut:testsTimeout "0"^^xsd:nonNegativeInteger ; rut:totalIndividualErrors "0"^^xsd:nonNegativeInteger ; prov:endedAtTime "2015-03-25T21:24:26.962Z"^^xsd:dateTime ; prov:startedAtTime "2015-03-25T21:24:24.399Z"^^xsd:dateTime ; prov:used ruts:4fb97fef-2acd-11b2-8041-aa98dd5aae63 ; prov:wasAssociatedWith schema: , http://xmlns.com/foaf/0.1/ ; prov:wasStartedBy http://localhost/ .

ruts:4fb97fef-2acd-11b2-8041-aa98dd5aae63 a rut:TestSuite , prov:Collection ; prov:hadMember .....

But if the tool finds errors the output is like that:

@Prefixes...

http://rdfunit.aksw.org/data/results#861d2bf8-2acd-11b2-80c0-aa98dd5aae63/861d62bc-2acd-11b2-80c0-aa98dd5aae63 a rut:TestCaseResult , rut:RLOGTestCaseResult , rlog:Entry , rut:ExtendedTestCaseResult , spin:ConstraintViolation ; rlog:level rlog:WARN ; rlog:message "http://schema.org/address is missing proper range" ; rlog:resource ; dcterms:date "2015-03-26T22:45:15.951Z"^^xsd:dateTime ; rut:testCase rutt:schema-RDFSRANGE-MISS-3f67d88cc38f44bb98e89a21653c7dc0f5227586b7b712f54cfdf78761762f2a ; spin:violationPath schema:address ; spin:violationRoot ; prov:wasGeneratedBy rutr:861d2bf8-2acd-11b2-80c0-aa98dd5aae63 .

rutr:861d2bf8-2acd-11b2-80c0-aa98dd5aae63 a prov:Activity , rut:TestExecution ; rut:source ; rut:testsError "0"^^xsd:nonNegativeInteger ; rut:testsFailed "3"^^xsd:nonNegativeInteger ; rut:testsRun "1594"^^xsd:nonNegativeInteger ; rut:testsSuceedded "1591"^^xsd:nonNegativeInteger ; rut:testsTimeout "0"^^xsd:nonNegativeInteger ; rut:totalIndividualErrors "1232"^^xsd:nonNegativeInteger ; prov:endedAtTime "2015-03-26T22:45:16.822Z"^^xsd:dateTime ; prov:startedAtTime "2015-03-26T22:45:14.576Z"^^xsd:dateTime ; prov:used ruts:861d634b-2acd-11b2-80c0-aa98dd5aae63 ; prov:wasAssociatedWith http://xmlns.com/foaf/0.1/ , schema: ; prov:wasStartedBy http://localhost/ .

http://rdfunit.aksw.org/data/results#861d2bf8-2acd-11b2-80c0-aa98dd5aae63/861d5bea-2acd-11b2-80c0-aa98dd5aae63 a rut:ExtendedTestCaseResult , rut:RLOGTestCaseResult , rlog:Entry , spin:ConstraintViolation , rut:TestCaseResult ; rlog:level rlog:WARN ; rlog:message "http://schema.org/address is missing proper range" ; rlog:resource ; dcterms:date "2015-03-26T22:45:15.95Z"^^xsd:dateTime ; rut:testCase rutt:schema-RDFSRANGE-MISS-3f67d88cc38f44bb98e89a21653c7dc0f5227586b7b712f54cfdf78761762f2a ; spin:violationPath schema:address ; spin:violationRoot ; prov:wasGeneratedBy rutr:861d2bf8-2acd-11b2-80c0-aa98dd5aae63 .

Is there a way to put the "summary" on the top of the file? after the prefixes?

— Reply to this email directly or view it on GitHub https://github.com/AKSW/RDFUnit/issues/35.

Kontokostas Dimitris

AndreAga commented 9 years ago

Hi Dimitris, thanks a lot :)

I noticed another strange thing.. I save the result in ttl format in a file, but I need to execute SPARQL query, so I create a Repository (with openRDF) and load the RDFUnit file. This file produces a parsing error when it tries to load this:

<http://rdfunit.aksw.org/data/results#9ed7af38-2acd-11b2-8088-aa98dd5aae63/9ed7e48a-2acd-11b2-8088-aa98dd5aae63>
        a                    rut:ExtendedTestCaseResult , rut:TestCaseResult , spin:ConstraintViolation , rlog:Entry , rut:RLOGTestCaseResult ;
        rlog:level           rlog:WARN ;
        rlog:message         "http://schema.org/address is missing proper range" ;
        rlog:resource        <Piazza Luigi Porro Lambertenghi, 14@it> ;
        dcterms:date         "2015-03-27T10:16:43.722Z"^^xsd:dateTime ;
        rut:testCase         rutt:schema-RDFSRANGE-MISS-3f67d88cc38f44bb98e89a21653c7dc0f5227586b7b712f54cfdf78761762f2a ;
        spin:violationPath   schema:address ;
        spin:violationRoot   <Piazza Luigi Porro Lambertenghi, 14@it> ;
        prov:wasGeneratedBy  rutr:9ed7af38-2acd-11b2-8088-aa98dd5aae63 .

and the error is:

Caused by: org.openrdf.rio.RDFParseException: IRI included an unencoded space: '32' [line 35]

because the parse encounters the error with:

rlog:resource  <Piazza Luigi Porro Lambertenghi, 14@it> ; ".

I think (maybe I'm wrong) that the rlog:resource must be a URI (without spaces, @ and other special characters) exactly as with rlog:ERROR:

<http://rdfunit.aksw.org/data/results#9ed7af38-2acd-11b2-8088-aa98dd5aae63/9ed7d395-2acd-11b2-8088-aa98dd5aae63>
        a                    rut:RLOGTestCaseResult , spin:ConstraintViolation , rlog:Entry , rut:TestCaseResult , rut:ExtendedTestCaseResult ;
        rlog:level           rlog:ERROR ;
        rlog:message         "http://schema.org/address contains a literal value" ;
        rlog:resource        <ee32301ce390cf0619bf799aa8525357> ;
        dcterms:date         "2015-03-27T10:16:43.107Z"^^xsd:dateTime ;
        rut:testCase         rutt:schema-OBJFUNC-18b7ef8b6d22121df33cceff06d08112af250d7df9fa514a30eb646d2fb8a94f ;
        spin:violationPath   schema:address ;
        spin:violationRoot   <ee32301ce390cf0619bf799aa8525357> ;
        spin:violationValue  "Via Torcegno, 19"@it ;
        prov:wasGeneratedBy  rutr:9ed7af38-2acd-11b2-8088-aa98dd5aae63 .

This last example doens't produce any error. What do you think?

jimkont commented 9 years ago

I always expect a resource for this but here we got a Literal. RDFUnit tries to convert that into a resource which results in something invalid. I have to fix this into the code and skip rlog:resource / spin:violationRoot in these cases

jimkont commented 9 years ago

I committed a fix, can you try?

AndreAga commented 9 years ago

I got:

Caused by: java.lang.AssertionError
    at org.aksw.rdfunit.tests.executors.ExtendedTestExecutor.executeSingleTest(ExtendedTestExecutor.java:81)
    at org.aksw.rdfunit.tests.executors.TestExecutor.execute(TestExecutor.java:108)
    at eu.unifiedviews.plugins.quality.rdfunit.RDFUnitValidation.validate(RDFUnitValidation.java:88)
    at eu.unifiedviews.plugins.quality.rdfunit.RDFUnit.innerExecute(RDFUnit.java:123)
    at eu.unifiedviews.helpers.dpu.exec.AbstractDpu.execute(AbstractDpu.java:117)

The line is this: assert result != null;

jimkont commented 9 years ago

Can you try again?

AndreAga commented 9 years ago

Thanks, now it works. I go on making some tests, if I will find other errors, I will report them here.