Exclusive group evaluation cause returns no result for q05

[mhoangvslev]

The command below will crash with NullPointerException

java -classpath "engines/FedX/target/FedX-1.0-SNAPSHOT.jar:engines/FedX/target/lib/*" org.example.FedX experiments/bsbm/benchmark/evaluation/ideal/config/batch_0/ideal.conf experiments/bsbm/benchmark/generation/q05/instance_1/injected.sparql /dev/null /dev/null /dev/null experiments/bsbm/benchmark/evaluation/ideal/q05/instance_1/batch_0/attempt_0/stats.csv experiments/bsbm/benchmark/evaluation/ideal/q05/instance_1/batch_0/provenance.csv

Hypothesis

• q05 was rewritten to take into account join order optimization (#18). The result is injected.sparql.opt below

  
  DEFINE sql:select-option "order"

PREFIX rdfs: http://www.w3.org/2000/01/rdf-schema# PREFIX rdf: http://www.w3.org/1999/02/22-rdf-syntax-ns# PREFIX bsbm: http://www4.wiwiss.fu-berlin.de/bizer/bsbm/v01/vocabulary/ PREFIX owl: http://www.w3.org/2002/07/owl#

SELECT DISTINCT ?product ?localProductLabel WHERE { {

const bsbm:Product92877

    ?localProductXYZ owl:sameAs bsbm:Product92877  .
    ?localProductXYZ bsbm:productFeature ?localProdFeatureXYZ . 
    ?localProdFeatureXYZ owl:sameAs ?prodFeature .
    ?localProductXYZ bsbm:productPropertyNumeric1 ?origProperty1  .
    ?localProductXYZ bsbm:productPropertyNumeric2 ?origProperty2  .
} .

{
    ?localProduct owl:sameAs ?product  .
    FILTER (bsbm:Product92877 != ?product)        
    ?localProduct rdfs:label ?localProductLabel  .
    ?localProduct bsbm:productFeature ?localProdFeature  .
    ?localProdFeature owl:sameAs ?prodFeature .
    ?localProduct bsbm:productPropertyNumeric1 ?simProperty1  .
    ?localProduct bsbm:productPropertyNumeric2 ?simProperty2  .    
} .

# Values are pre-determined because we knew the boundaries from the normal distribution
FILTER(?simProperty1 < (?origProperty1 + 20) && ?simProperty1 > (?origProperty1 - 20))
FILTER(?simProperty2 < (?origProperty2 + 70) && ?simProperty2 > (?origProperty2 - 70))

}

ORDER BY ?localProductLabel

LIMIT 5

- However, the corresponding optimized version `provenance.sparql.opt` were not up-to-date, i.e, the order of triple patterns is not **the same as** in ìnjected.sparql.opt`, causing the injection engine to assign the wrong sources to the wrong triple pattern.

# Solution:
- [x] Short term: rewrite the order of triple patterns in  `provenance.sparql.opt` so that they are **the same as** in ìnjected.sparql.opt`. Re-run generation workflow then re-evaluate.

[mhoangvslev]

Problem still persist with the correct provenance.sparql.opt:

Diagnosis:

• Given the source selection below (experiments/bsbm/benchmark/evaluation/ideal/q05/instance_8/batch_0/provenance.csv)

tp1	tp2	tp3	tp4	tp5	tp6	tp7	tp8	tp9	tp10	tp11
http://www.ratingsite2.fr/	http://www.ratingsite2.fr/	http://www.ratingsite2.fr/	http://www.ratingsite2.fr/	http://www.ratingsite2.fr/	http://www.ratingsite2.fr/	http://www.ratingsite6.fr/	http://www.ratingsite6.fr/	http://www.ratingsite6.fr/	http://www.ratingsite6.fr/	http://www.ratingsite6.fr/
http://www.ratingsite2.fr/	http://www.ratingsite2.fr/	http://www.ratingsite2.fr/	http://www.ratingsite2.fr/	http://www.ratingsite2.fr/	http://www.ratingsite2.fr/	http://www.vendor5.fr/	http://www.vendor5.fr/	http://www.vendor5.fr/	http://www.vendor5.fr/	http://www.vendor5.fr/

• FedX when forced, will produce this plan:

QueryRoot
   Slice (limit=5)
      Distinct
         Projection
            ProjectionElemList
               ProjectionElem "product"
               ProjectionElem "localProductLabel"
                  (...)
                  NJoin
                     ExclusiveGroup
                        ExclusiveStatement
                           Var (name=localProduct)
                           Var (name=_const_9285ccfc_uri, value=http://www.w3.org/2000/01/rdf-schema#label, anonymous)
                           Var (name=localProductLabel)
                           StatementSource (id=sparql_www.ratingsite2.fr_, type=REMOTE)
                        ExclusiveStatement
                           Var (name=localProduct)
                           Var (name=_const_e248e789_uri, value=http://www4.wiwiss.fu-berlin.de/bizer/bsbm/v01/vocabulary/productFeature, anonymous)
                           Var (name=localProdFeature)
                           StatementSource (id=sparql_www.ratingsite2.fr_, type=REMOTE)
                        ExclusiveStatement
                           Var (name=localProduct)
                           Var (name=_const_ac29ec86_uri, value=http://www4.wiwiss.fu-berlin.de/bizer/bsbm/v01/vocabulary/productPropertyNumeric1, anonymous)
                           Var (name=simProperty1)
                           StatementSource (id=sparql_www.ratingsite2.fr_, type=REMOTE)
                        ExclusiveStatement
                           Var (name=localProduct)
                           Var (name=_const_ac29ec87_uri, value=http://www4.wiwiss.fu-berlin.de/bizer/bsbm/v01/vocabulary/productPropertyNumeric2, anonymous)
                           Var (name=simProperty2)
                           StatementSource (id=sparql_www.ratingsite2.fr_, type=REMOTE)
                        ExclusiveStatement
                           Var (name=localProduct)
                           Var (name=_const_9f24f144_uri, value=http://www.w3.org/2002/07/owl#sameAs, anonymous)
                           Var (name=product)
                           StatementSource (id=sparql_www.ratingsite2.fr_, type=REMOTE)
                        ExclusiveStatement
                           Var (name=localProdFeature)
                           Var (name=_const_9f24f144_uri, value=http://www.w3.org/2002/07/owl#sameAs, anonymous)
                           Var (name=prodFeature)
                           StatementSource (id=sparql_www.ratingsite2.fr_, type=REMOTE)
                        FilterExpr
                           Compare (!=)
                              ValueConstant (value=http://www4.wiwiss.fu-berlin.de/bizer/bsbm/v01/vocabulary/Product128382)
                              Var (name=product)
                     StatementSourcePattern
                        Var (name=localProductXYZ)
                        Var (name=_const_9f24f144_uri, value=http://www.w3.org/2002/07/owl#sameAs, anonymous)
                        Var (name=_const_8c12c1a3_uri, value=http://www4.wiwiss.fu-berlin.de/bizer/bsbm/v01/vocabulary/Product128382, anonymous)
                        StatementSource (id=sparql_www.ratingsite6.fr_, type=REMOTE)
                        StatementSource (id=sparql_www.vendor5.fr_, type=REMOTE)
                     StatementSourcePattern
                        Var (name=localProductXYZ)
                        Var (name=_const_e248e789_uri, value=http://www4.wiwiss.fu-berlin.de/bizer/bsbm/v01/vocabulary/productFeature, anonymous)
                        Var (name=localProdFeatureXYZ)
                        StatementSource (id=sparql_www.ratingsite6.fr_, type=REMOTE)
                        StatementSource (id=sparql_www.vendor5.fr_, type=REMOTE)
                     StatementSourcePattern
                        Var (name=localProdFeatureXYZ)
                        Var (name=_const_9f24f144_uri, value=http://www.w3.org/2002/07/owl#sameAs, anonymous)
                        Var (name=prodFeature)
                        StatementSource (id=sparql_www.ratingsite6.fr_, type=REMOTE)
                        StatementSource (id=sparql_www.vendor5.fr_, type=REMOTE)
                     StatementSourcePattern
                        Var (name=localProductXYZ)
                        Var (name=_const_ac29ec86_uri, value=http://www4.wiwiss.fu-berlin.de/bizer/bsbm/v01/vocabulary/productPropertyNumeric1, anonymous)
                        Var (name=origProperty1)
                        StatementSource (id=sparql_www.ratingsite6.fr_, type=REMOTE)
                        StatementSource (id=sparql_www.vendor5.fr_, type=REMOTE)
                     StatementSourcePattern
                        Var (name=localProductXYZ)
                        Var (name=_const_ac29ec87_uri, value=http://www4.wiwiss.fu-berlin.de/bizer/bsbm/v01/vocabulary/productPropertyNumeric2, anonymous)
                        Var (name=origProperty2)
                        StatementSource (id=sparql_www.ratingsite6.fr_, type=REMOTE)
                        StatementSource (id=sparql_www.vendor5.fr_, type=REMOTE)

• The bgp1 was evaluated exclusively on www.ratingsite2.fr, which is correct. The each tp of bgp2 is evaluated on ratingsite6 and vendor5 which is also correct.
• When removing either source for bgp2, the bgp2 is evaluated exclusively on the remaining source, meaning that both ratingsite6 and vendor5 yield results.
• The debugger halts at while (res.hasNext())
• Using LIMIT=1 produce results

[mhoangvslev]

The experiments/bsbm/benchmark/evaluation/ideal/q05/instance_5/batch_0/provenance.csv has similar exclusive group pattern, but for bgp2 and it works

tp1	tp2	tp3	tp4	tp5	tp6	tp7	tp8	tp9	tp10	tp11
http://www.ratingsite7.fr/	http://www.ratingsite7.fr/	http://www.ratingsite7.fr/	http://www.ratingsite7.fr/	http://www.ratingsite7.fr/	http://www.ratingsite7.fr/	http://www.ratingsite6.fr/	http://www.ratingsite6.fr/	http://www.ratingsite6.fr/	http://www.ratingsite6.fr/	http://www.ratingsite6.fr/
http://www.ratingsite5.fr/	http://www.ratingsite5.fr/	http://www.ratingsite5.fr/	http://www.ratingsite5.fr/	http://www.ratingsite5.fr/	http://www.ratingsite5.fr/	http://www.ratingsite6.fr/	http://www.ratingsite6.fr/	http://www.ratingsite6.fr/	http://www.ratingsite6.fr/	http://www.ratingsite6.fr/

[mhoangvslev]

For now, temporarily disable the optimisation query (#18 ) to see if the issue persists. If it does, then the issue most likely caused by query.py/unwrap()