Semagrow: q03 and q07 do not work

[Yotlan]

Weird behavior

When you execute these 2 queries with the following command:

./semagrow.sh repository.ttl http://localhost:34209/sparql ../../test/results.csv ../../test/source_selection.txt ../../test/query_plan.txt ../../test/stats.csv 120 ../../experiments/bsbm/benchmark/generation/q07/instance_0/injected.sparql 0 false true metadata.ttl

You get a working query who return an empty result ... because of the sameAs predicate inside an OPTIONAL clause. For the query 3 is weird because you get an error when you execute the following command:

./semagrow.sh repository.ttl http://localhost:34209/sparql ../../test/results.csv ../../test/source_selection.txt ../../test/query_plan.txt ../../test/stats.csv 120 ../../experiments/bsbm/benchmark/generation/q07/instance_0/injected.sparql 0 false true metadata.ttl

The corresponding stack trace we get:

Execute query ../../experiments/bsbm/benchmark/generation/q03/instance_0/injected.sparql for 20 endpoints...
mvn exec:java -pl rdf4j/ -Dexec.mainClass=org.semagrow.cli.CliMain -Dexec.args=repository.ttl http://localhost:34209/sparql ../../test/results.csv ../../test/source_selection.txt ../../test/query_plan.txt ../../test/stats.csv 120 metadata.ttl ../../experiments/bsbm/benchmark/generation/q03/instance_0/injected.sparql http://www.ratingsite0.fr/ http://www.ratingsite1.fr/ http://www.ratingsite2.fr/ http://www.ratingsite3.fr/ http://www.ratingsite4.fr/ http://www.ratingsite5.fr/ http://www.ratingsite6.fr/ http://www.ratingsite7.fr/ http://www.ratingsite8.fr/ http://www.ratingsite9.fr/ http://www.vendor0.fr/ http://www.vendor1.fr/ http://www.vendor2.fr/ http://www.vendor3.fr/ http://www.vendor4.fr/ http://www.vendor5.fr/ http://www.vendor6.fr/ http://www.vendor7.fr/ http://www.vendor8.fr/ http://www.vendor9.fr/
[INFO] Scanning for projects...
[INFO] 
[INFO] --------------------< org.semagrow:semagrow-rdf4j >---------------------
[INFO] Building SemaGrow RDF4J Repository 2.2.1
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] 
[INFO] --- exec-maven-plugin:3.1.0:java (default-cli) @ semagrow-rdf4j ---
[org.semagrow.cli.CliMain.main()] INFO org.semagrow.cli.CliMain - repository.ttl
{endpoint=http://localhost:34209/sparql}
[org.semagrow.cli.CliMain.main()] INFO org.semagrow.repository.SemagrowRepositoryResolver - /GDD/RSFB/engines/semagrow/repository.ttl
[org.semagrow.cli.CliMain.main()] INFO org.semagrow.repository.SemagrowRepositoryResolver - Optional[Turtle (mimeTypes=text/turtle, application/x-turtle; ext=ttl)]
[org.semagrow.cli.CliMain.main()] INFO org.semagrow.cli.CliMain - ../../experiments/bsbm/benchmark/generation/q03/instance_0/injected.sparql
[org.semagrow.cli.CliMain.main()] INFO org.semagrow.cli.CliMain - PREFIX xsd: <http://www.w3.org/2001/XMLSchema#>
PREFIX bsbm: <http://www4.wiwiss.fu-berlin.de/bizer/bsbm/v01/vocabulary/>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
PREFIX owl: <http://www.w3.org/2002/07/owl#>

SELECT DISTINCT ?product ?label WHERE {
    ?localProduct owl:sameAs ?product .
    ?localProduct rdfs:label ?label .
    # const bsbm:ProductType532
    ?localProduct rdf:type ?localProductType .
    ?localProductType owl:sameAs bsbm:ProductType532 .

    # const bsbm:ProductFeature18075
    ?localProduct bsbm:productFeature ?localProductFeature1 .
    ?localProductFeature1 owl:sameAs bsbm:ProductFeature18075 .
    ?localProduct bsbm:productPropertyNumeric1 ?p1 .
    # const "1125"^^xsd:integer < ?p1 
    FILTER ( ?p1 > "1125"^^xsd:integer ) 
    ?localProduct bsbm:productPropertyNumeric3 ?p3 .
    # const "1369"^^xsd:integer > ?p3
    FILTER (?p3 < "1369"^^xsd:integer )

    OPTIONAL { 
        # const!* not bsbm:ProductFeature22096
        ?localProduct bsbm:productFeature ?localProductFeature2 .
        ?localProductFeature2 owl:sameAs bsbm:ProductFeature22096 .
        ?localProduct rdfs:label ?testVar 
    }
    FILTER (!bound(?testVar)) 
}
ORDER BY ?product ?label
LIMIT 10
[org.semagrow.cli.CliMain.main()] INFO org.semagrow.cli.CliMain - ../../test/results.csv
[org.semagrow.cli.CliMain.main()] INFO org.semagrow.cli.CliMain - Evaluating query ../../experiments/bsbm/benchmark/generation/q03/instance_0/injected.sparql
[org.semagrow.cli.CliMain.main()] INFO org.semagrow.sail.SemagrowSailConnection - [InternalReactive] Starting decomposition of QueryRoot
   Slice ( limit=10 )
      Distinct
         Projection
            ProjectionElemList
               ProjectionElem "product"
               ProjectionElem "label"
            Order
               OrderElem (ASC)
                  Var (name=product)
               OrderElem (ASC)
                  Var (name=label)
               Filter
                  Not
                     Bound
                        Var (name=testVar)
                  Filter
                     Compare (<)
                        Var (name=p3)
                        ValueConstant (value="1369"^^<http://www.w3.org/2001/XMLSchema#integer>)
                     Filter
                        Compare (>)
                           Var (name=p1)
                           ValueConstant (value="1125"^^<http://www.w3.org/2001/XMLSchema#integer>)
                        LeftJoin
                           Join
                              Join
                                 Join
                                    Join
                                       Join
                                          Join
                                             Join
                                                StatementPattern
                                                   Var (name=localProduct)
                                                   Var (name=_const_9f24f144_uri, value=http://www.w3.org/2002/07/owl#sameAs, anonymous)
                                                   Var (name=product)
                                                StatementPattern
                                                   Var (name=localProduct)
                                                   Var (name=_const_9285ccfc_uri, value=http://www.w3.org/2000/01/rdf-schema#label, anonymous)
                                                   Var (name=label)
                                             StatementPattern
                                                Var (name=localProduct)
                                                Var (name=_const_f5e5585a_uri, value=http://www.w3.org/1999/02/22-rdf-syntax-ns#type, anonymous)
                                                Var (name=localProductType)
                                          StatementPattern
                                             Var (name=localProductType)
                                             Var (name=_const_9f24f144_uri, value=http://www.w3.org/2002/07/owl#sameAs, anonymous)
                                             Var (name=_const_ae13748d_uri, value=http://www4.wiwiss.fu-berlin.de/bizer/bsbm/v01/vocabulary/ProductType532, anonymous)
                                       StatementPattern
                                          Var (name=localProduct)
                                          Var (name=_const_e248e789_uri, value=http://www4.wiwiss.fu-berlin.de/bizer/bsbm/v01/vocabulary/productFeature, anonymous)
                                          Var (name=localProductFeature1)
                                    StatementPattern
                                       Var (name=localProductFeature1)
                                       Var (name=_const_9f24f144_uri, value=http://www.w3.org/2002/07/owl#sameAs, anonymous)
                                       Var (name=_const_b177dd7e_uri, value=http://www4.wiwiss.fu-berlin.de/bizer/bsbm/v01/vocabulary/ProductFeature18075, anonymous)
                                 StatementPattern
                                    Var (name=localProduct)
                                    Var (name=_const_ac29ec86_uri, value=http://www4.wiwiss.fu-berlin.de/bizer/bsbm/v01/vocabulary/productPropertyNumeric1, anonymous)
                                    Var (name=p1)
                              StatementPattern
                                 Var (name=localProduct)
                                 Var (name=_const_ac29ec88_uri, value=http://www4.wiwiss.fu-berlin.de/bizer/bsbm/v01/vocabulary/productPropertyNumeric3, anonymous)
                                 Var (name=p3)
                           Join
                              Join
                                 StatementPattern
                                    Var (name=localProduct)
                                    Var (name=_const_e248e789_uri, value=http://www4.wiwiss.fu-berlin.de/bizer/bsbm/v01/vocabulary/productFeature, anonymous)
                                    Var (name=localProductFeature2)
                                 StatementPattern
                                    Var (name=localProductFeature2)
                                    Var (name=_const_9f24f144_uri, value=http://www.w3.org/2002/07/owl#sameAs, anonymous)
                                    Var (name=_const_b1833b04_uri, value=http://www4.wiwiss.fu-berlin.de/bizer/bsbm/v01/vocabulary/ProductFeature22096, anonymous)
                              StatementPattern
                                 Var (name=localProduct)
                                 Var (name=_const_9285ccfc_uri, value=http://www.w3.org/2000/01/rdf-schema#label, anonymous)
                                 Var (name=testVar)

[org.semagrow.cli.CliMain.main()] INFO org.semagrow.sail.SemagrowSailConnection - tupleExpr: QueryRoot
   Slice ( limit=10 )
      Distinct
         Projection
            ProjectionElemList
               ProjectionElem "product"
               ProjectionElem "label"
            Order
               OrderElem (ASC)
                  Var (name=product)
               OrderElem (ASC)
                  Var (name=label)
               Filter
                  Not
                     Bound
                        Var (name=testVar)
                  Filter
                     Compare (<)
                        Var (name=p3)
                        ValueConstant (value="1369"^^<http://www.w3.org/2001/XMLSchema#integer>)
                     Filter
                        Compare (>)
                           Var (name=p1)
                           ValueConstant (value="1125"^^<http://www.w3.org/2001/XMLSchema#integer>)
                        LeftJoin
                           Join
                              Join
                                 Join
                                    Join
                                       Join
                                          Join
                                             Join
                                                StatementPattern
                                                   Var (name=localProduct)
                                                   Var (name=_const_9f24f144_uri, value=http://www.w3.org/2002/07/owl#sameAs, anonymous)
                                                   Var (name=product)
                                                StatementPattern
                                                   Var (name=localProduct)
                                                   Var (name=_const_9285ccfc_uri, value=http://www.w3.org/2000/01/rdf-schema#label, anonymous)
                                                   Var (name=label)
                                             StatementPattern
                                                Var (name=localProduct)
                                                Var (name=_const_f5e5585a_uri, value=http://www.w3.org/1999/02/22-rdf-syntax-ns#type, anonymous)
                                                Var (name=localProductType)
                                          StatementPattern
                                             Var (name=localProductType)
                                             Var (name=_const_9f24f144_uri, value=http://www.w3.org/2002/07/owl#sameAs, anonymous)
                                             Var (name=_const_ae13748d_uri, value=http://www4.wiwiss.fu-berlin.de/bizer/bsbm/v01/vocabulary/ProductType532, anonymous)
                                       StatementPattern
                                          Var (name=localProduct)
                                          Var (name=_const_e248e789_uri, value=http://www4.wiwiss.fu-berlin.de/bizer/bsbm/v01/vocabulary/productFeature, anonymous)
                                          Var (name=localProductFeature1)
                                    StatementPattern
                                       Var (name=localProductFeature1)
                                       Var (name=_const_9f24f144_uri, value=http://www.w3.org/2002/07/owl#sameAs, anonymous)
                                       Var (name=_const_b177dd7e_uri, value=http://www4.wiwiss.fu-berlin.de/bizer/bsbm/v01/vocabulary/ProductFeature18075, anonymous)
                                 StatementPattern
                                    Var (name=localProduct)
                                    Var (name=_const_ac29ec86_uri, value=http://www4.wiwiss.fu-berlin.de/bizer/bsbm/v01/vocabulary/productPropertyNumeric1, anonymous)
                                    Var (name=p1)
                              StatementPattern
                                 Var (name=localProduct)
                                 Var (name=_const_ac29ec88_uri, value=http://www4.wiwiss.fu-berlin.de/bizer/bsbm/v01/vocabulary/productPropertyNumeric3, anonymous)
                                 Var (name=p3)
                           Join
                              Join
                                 StatementPattern
                                    Var (name=localProduct)
                                    Var (name=_const_e248e789_uri, value=http://www4.wiwiss.fu-berlin.de/bizer/bsbm/v01/vocabulary/productFeature, anonymous)
                                    Var (name=localProductFeature2)
                                 StatementPattern
                                    Var (name=localProductFeature2)
                                    Var (name=_const_9f24f144_uri, value=http://www.w3.org/2002/07/owl#sameAs, anonymous)
                                    Var (name=_const_b1833b04_uri, value=http://www4.wiwiss.fu-berlin.de/bizer/bsbm/v01/vocabulary/ProductFeature22096, anonymous)
                              StatementPattern
                                 Var (name=localProduct)
                                 Var (name=_const_9285ccfc_uri, value=http://www.w3.org/2000/01/rdf-schema#label, anonymous)
                                 Var (name=testVar)

dataset: null
bindings: []
includeOnlySources: []
excludeSources: []
[org.semagrow.cli.CliMain.main()] INFO org.semagrow.sail.SemagrowSailConnection - decomposer: org.semagrow.plan.SimpleQueryCompiler@d7926e1
java.lang.ClassCastException: class org.semagrow.plan.queryblock.Quantifier$Var cannot be cast to class org.eclipse.rdf4j.query.algebra.Var (org.semagrow.plan.queryblock.Quantifier$Var and org.eclipse.rdf4j.query.algebra.Var are in unnamed module of loader org.codehaus.mojo.exec.URLClassLoaderBuilder$ExecJavaClassLoader @16eedaa6)
        at org.eclipse.rdf4j.query.algebra.Bound.replaceChildNode(Bound.java:75)
        at org.eclipse.rdf4j.query.algebra.AbstractQueryModelNode.replaceWith(AbstractQueryModelNode.java:91)
        at org.semagrow.plan.queryblock.QueryBlockBuilder$VarReplacer.meet(QueryBlockBuilder.java:435)
        at org.eclipse.rdf4j.query.algebra.Var.visit(Var.java:79)
        at org.eclipse.rdf4j.query.algebra.Bound.visitChildren(Bound.java:69)
        at org.eclipse.rdf4j.query.algebra.helpers.AbstractQueryModelVisitor.meetNode(AbstractQueryModelVisitor.java:560)
        at org.eclipse.rdf4j.query.algebra.helpers.AbstractQueryModelVisitor.meet(AbstractQueryModelVisitor.java:138)
        at org.eclipse.rdf4j.query.algebra.Bound.visit(Bound.java:64)
        at org.eclipse.rdf4j.query.algebra.UnaryValueOperator.visitChildren(UnaryValueOperator.java:70)
        at org.eclipse.rdf4j.query.algebra.helpers.AbstractQueryModelVisitor.meetNode(AbstractQueryModelVisitor.java:560)
        at org.eclipse.rdf4j.query.algebra.helpers.AbstractQueryModelVisitor.meetUnaryValueOperator(AbstractQueryModelVisitor.java:590)
        at org.eclipse.rdf4j.query.algebra.helpers.AbstractQueryModelVisitor.meet(AbstractQueryModelVisitor.java:378)
        at org.eclipse.rdf4j.query.algebra.Not.visit(Not.java:32)
        at org.semagrow.plan.queryblock.QueryBlockBuilder$VarReplacer.process(QueryBlockBuilder.java:444)
        at org.semagrow.plan.queryblock.QueryBlockBuilder.meet(QueryBlockBuilder.java:295)
        at org.eclipse.rdf4j.query.algebra.Filter.visit(Filter.java:69)
        at org.eclipse.rdf4j.query.algebra.UnaryTupleOperator.visitChildren(UnaryTupleOperator.java:91)
        at org.eclipse.rdf4j.query.algebra.Order.visitChildren(Order.java:87)
        at org.eclipse.rdf4j.query.algebra.helpers.AbstractQueryModelVisitor.meetNode(AbstractQueryModelVisitor.java:560)
        at org.eclipse.rdf4j.query.algebra.helpers.AbstractQueryModelVisitor.meetUnaryTupleOperator(AbstractQueryModelVisitor.java:580)
        at org.eclipse.rdf4j.query.algebra.helpers.AbstractQueryModelVisitor.meet(AbstractQueryModelVisitor.java:388)
        at org.semagrow.plan.queryblock.QueryBlockBuilder.meet(QueryBlockBuilder.java:139)
        at org.eclipse.rdf4j.query.algebra.Order.visit(Order.java:78)
        at org.eclipse.rdf4j.query.algebra.UnaryTupleOperator.visitChildren(UnaryTupleOperator.java:91)
        at org.eclipse.rdf4j.query.algebra.Projection.visitChildren(Projection.java:81)
        at org.eclipse.rdf4j.query.algebra.helpers.AbstractQueryModelVisitor.meetNode(AbstractQueryModelVisitor.java:560)
        at org.eclipse.rdf4j.query.algebra.helpers.AbstractQueryModelVisitor.meetUnaryTupleOperator(AbstractQueryModelVisitor.java:580)
        at org.eclipse.rdf4j.query.algebra.helpers.AbstractQueryModelVisitor.meet(AbstractQueryModelVisitor.java:398)
        at org.semagrow.plan.queryblock.QueryBlockBuilder.meet(QueryBlockBuilder.java:188)
        at org.eclipse.rdf4j.query.algebra.Projection.visit(Projection.java:75)
        at org.eclipse.rdf4j.query.algebra.UnaryTupleOperator.visitChildren(UnaryTupleOperator.java:91)
        at org.eclipse.rdf4j.query.algebra.helpers.AbstractQueryModelVisitor.meetNode(AbstractQueryModelVisitor.java:560)
        at org.eclipse.rdf4j.query.algebra.helpers.AbstractQueryModelVisitor.meetUnaryTupleOperator(AbstractQueryModelVisitor.java:580)
        at org.eclipse.rdf4j.query.algebra.helpers.AbstractQueryModelVisitor.meet(AbstractQueryModelVisitor.java:203)
        at org.semagrow.plan.queryblock.QueryBlockBuilder.meet(QueryBlockBuilder.java:110)
        at org.eclipse.rdf4j.query.algebra.Distinct.visit(Distinct.java:29)
        at org.eclipse.rdf4j.query.algebra.UnaryTupleOperator.visitChildren(UnaryTupleOperator.java:91)
        at org.eclipse.rdf4j.query.algebra.helpers.AbstractQueryModelVisitor.meetNode(AbstractQueryModelVisitor.java:560)
        at org.eclipse.rdf4j.query.algebra.helpers.AbstractQueryModelVisitor.meetUnaryTupleOperator(AbstractQueryModelVisitor.java:580)
        at org.eclipse.rdf4j.query.algebra.helpers.AbstractQueryModelVisitor.meet(AbstractQueryModelVisitor.java:448)
        at org.semagrow.plan.queryblock.QueryBlockBuilder.meet(QueryBlockBuilder.java:169)
        at org.eclipse.rdf4j.query.algebra.Slice.visit(Slice.java:85)
        at org.eclipse.rdf4j.query.algebra.UnaryTupleOperator.visitChildren(UnaryTupleOperator.java:91)
        at org.eclipse.rdf4j.query.algebra.helpers.AbstractQueryModelVisitor.meetNode(AbstractQueryModelVisitor.java:560)
        at org.eclipse.rdf4j.query.algebra.helpers.AbstractQueryModelVisitor.meet(AbstractQueryModelVisitor.java:413)
        at org.eclipse.rdf4j.query.algebra.QueryRoot.visit(QueryRoot.java:45)
        at org.semagrow.plan.queryblock.QueryBlockBuilder.build(QueryBlockBuilder.java:69)
        at org.semagrow.plan.SimpleQueryCompiler.blockify(SimpleQueryCompiler.java:106)
        at org.semagrow.plan.SimpleQueryCompiler.compile_aroundBody0(SimpleQueryCompiler.java:59)
        at org.semagrow.plan.SimpleQueryCompiler$AjcClosure1.run(SimpleQueryCompiler.java:1)
        at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167)
        at org.semagrow.art.MethodLogger.around(MethodLogger.java:31)
        at org.semagrow.plan.SimpleQueryCompiler.compile(SimpleQueryCompiler.java:53)
        at org.semagrow.sail.SemagrowSailConnection.decompose(SemagrowSailConnection.java:234)
        at org.semagrow.sail.SemagrowSailConnection.evaluateInternalReactive(SemagrowSailConnection.java:164)
        at org.semagrow.sail.SemagrowSailConnection.evaluateReactive(SemagrowSailConnection.java:147)
        at org.semagrow.query.impl.SemagrowSailTupleQuery.evaluate(SemagrowSailTupleQuery.java:79)
        at org.semagrow.cli.CliMain.main(CliMain.java:89)
        at org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:279)
        at java.base/java.lang.Thread.run(Thread.java:833)
[org.semagrow.cli.CliMain.main()] INFO org.semagrow.sail.SemagrowSailConnection - Decomposition duration = 168
[WARNING] 
java.lang.NullPointerException: Cannot invoke "Object.toString()" because "decomposed" is null
    at org.semagrow.sail.SemagrowSailConnection.evaluateInternalReactive (SemagrowSailConnection.java:167)
    at org.semagrow.sail.SemagrowSailConnection.evaluateReactive (SemagrowSailConnection.java:147)
    at org.semagrow.query.impl.SemagrowSailTupleQuery.evaluate (SemagrowSailTupleQuery.java:79)
    at org.semagrow.cli.CliMain.main (CliMain.java:89)
    at org.codehaus.mojo.exec.ExecJavaMojo$1.run (ExecJavaMojo.java:279)
    at java.lang.Thread.run (Thread.java:833)
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  13.371 s
[INFO] Finished at: 2023-04-18T15:21:03+02:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:3.1.0:java (default-cli) on project semagrow-rdf4j: An exception occurred while executing the Java class. Cannot invoke "Object.toString()" because "decomposed" is null -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
Execute query ../../experiments/bsbm/benchmark/generation/q03/instance_0/injected.sparql for 20 endpoints is done!

And when you want to fix this bug by replacing this line 435 in RSFB/engines/semagrow/core/src/main/java/org/semagrow/plan/queryblock/QueryBlockBuilder.java:

var.replaceWith(qvar.get());

by these lines:

Quantifier.Var tqvar = qvar.get();
var.replaceWith(new Var(tqvar.getName()));

This query work but return an empty result ..... the same for all queries.

[mhoangvslev]

I debugged q03 and found these interesting clues:

• p1, p3 and testVar are accompanied by FILTER clause
• p1 and p3 have binary operator inside FILTER and the said exception was not raised.
• testVar has !bound() inside FILTER and the exception was raised.

My guess is that bound() is not supported, therefore you have to implement it. However it is not within the scope of the project, thus you can let it slide.

[mhoangvslev]

Semagrow uses RDF4J, which has bound function. https://rdf4j.org/javadoc/3.3.0/org/eclipse/rdf4j/query/algebra/Bound.html