search

VirtualFlyBrain / vfb-pipeline-collectdata

Apache License 2.0
0 stars 0 forks source link

Reactivate Validation step #14

Open matentzn opened 3 years ago

matentzn commented 3 years ago

Dont forget

matentzn commented 3 years ago

ENVIRONMENT VARIABLE in container -> make sure shacl checks are run.

Robbie1977 commented 3 years ago

set VALIDATESHACL=true in rancher job so will run on next build

Robbie1977 commented 3 years ago

Collect data just stops after 'Validating KB with SHACL..' https://jenkins.virtualflybrain.org/view/pip_pipeline2/job/pip_vfb-triplestore/lastFailedBuild/console

@dosumis I assume this is the test configs are missing!?

matentzn commented 3 years ago

My guess:

https://github.com/VirtualFlyBrain/vfb-pipeline-collectdata/blob/41b00ae876860c3f476c3ac4bd9ffe10eec23d6b/process.sh#L161

fails and writes something to validation.txt which is relevant for debugging. Maybe this shacl test fails for some reason..

Robbie1977 commented 3 years ago 
@prefix owl:   <http://www.w3.org/2002/07/owl#> .
@prefix rdf:   <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
@prefix xml:   <http://www.w3.org/XML/1998/namespace> .
@prefix xsd:   <http://www.w3.org/2001/XMLSchema#> .
@prefix rdfs:  <http://www.w3.org/2000/01/rdf-schema#> .

[ a       <http://www.w3.org/ns/shacl#ValidationReport> ;
  <http://www.w3.org/ns/shacl#conforms>
          false ;
  <http://www.w3.org/ns/shacl#result>
          [ a       <http://www.w3.org/ns/shacl#ValidationResult> ;
            <http://www.w3.org/ns/shacl#focusNode>
                    <http://purl.obolibrary.org/obo/FBbt_00050095> ;
            <http://www.w3.org/ns/shacl#resultMessage>
                    "Property needs to have at least 1 values, but found 0" ;
            <http://www.w3.org/ns/shacl#resultPath>
                    [ <http://www.w3.org/ns/shacl#inversePath>
                              rdf:type ] ;
            <http://www.w3.org/ns/shacl#resultSeverity>
                    <http://www.w3.org/ns/shacl#Violation> ;
            <http://www.w3.org/ns/shacl#sourceConstraintComponent>
                    <http://www.w3.org/ns/shacl#MinCountConstraintComponent> ;
            <http://www.w3.org/ns/shacl#sourceShape>
                    [] 
          ]
] .

http://virtualflybrain.org/data/VFB/OWL/out/validation.txt

Robbie1977 commented 3 years ago

The test looks wrong as FBbt:00050095 = adult ALad1 lineage clone seems like an oddly specific check

@prefix dash: <http://datashapes.org/dash#> .
@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .
@prefix vfb: <http://virtualflybrain.org/> .
@prefix sh: <http://www.w3.org/ns/shacl#> .
@prefix xsd: <http://www.w3.org/2001/XMLSchema#> .
@prefix FBbt:  <http://purl.obolibrary.org/obo/FBbt_> .

vfb:DataSetCountShape
            a sh:NodeShape ;
            sh:targetNode FBbt:00050095 ;
            sh:property [
                    sh:path [ sh:inversePath rdf:type ] ;
                    sh:minCount 1 ;
            ] .
matentzn commented 3 years ago

I agree. For now, better deactivate validation until @dosumis had time to find a check that actually passes!