Evaluation on FedX doens't give the same result as Virtuoso

[mhoangvslev]

Given the experiments/bsbm/benchmark/generation/q05/instance_4/injected.sparql.opt:

DEFINE sql:select-option "order"

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
PREFIX bsbm: <http://www4.wiwiss.fu-berlin.de/bizer/bsbm/v01/vocabulary/>
PREFIX owl: <http://www.w3.org/2002/07/owl#>

SELECT DISTINCT ?product ?localProductLabel
WHERE {
    {
        # const bsbm:Product133677
        ?localProductXYZ owl:sameAs bsbm:Product133677  .
        ?localProductXYZ bsbm:productFeature ?localProdFeatureXYZ . 
        ?localProdFeatureXYZ owl:sameAs ?prodFeature .
        ?localProductXYZ bsbm:productPropertyNumeric1 ?origProperty1  .
        ?localProductXYZ bsbm:productPropertyNumeric2 ?origProperty2  .
    } .

    {
        ?localProduct owl:sameAs ?product  .
        FILTER (bsbm:Product133677 != ?product)        
        ?localProduct rdfs:label ?localProductLabel  .
        ?localProduct bsbm:productFeature ?localProdFeature  .
        ?localProdFeature owl:sameAs ?prodFeature .
        ?localProduct bsbm:productPropertyNumeric1 ?simProperty1  .
        ?localProduct bsbm:productPropertyNumeric2 ?simProperty2  .    
    } .

    # Values are pre-determined because we knew the boundaries from the normal distribution
    FILTER(?simProperty1 < (?origProperty1 + 20) && ?simProperty1 > (?origProperty1 - 20))
    FILTER(?simProperty2 < (?origProperty2 + 70) && ?simProperty2 > (?origProperty2 - 70))

}
# ORDER BY ?localProductLabel
LIMIT 5

• FedX (experiments/bsbm/benchmark/evaluation/fedx/q05/instance_4/batch_0/results.csv) yields the results below:

product	localProductLabel
http://www4.wiwiss.fu-berlin.de/bizer/bsbm/v01/vocabulary/Product97476	befogs
http://www4.wiwiss.fu-berlin.de/bizer/bsbm/v01/vocabulary/Product180800	directs
http://www4.wiwiss.fu-berlin.de/bizer/bsbm/v01/vocabulary/Product156288	overhauls
http://www4.wiwiss.fu-berlin.de/bizer/bsbm/v01/vocabulary/Product99042	vesper's
http://www4.wiwiss.fu-berlin.de/bizer/bsbm/v01/vocabulary/Product74631	piped

• Virtuoso (experiments/bsbm/benchmark/generation/q05/instance_4/batch_0/results.csv) yields the results below:

product	localProductLabel
http://www4.wiwiss.fu-berlin.de/bizer/bsbm/v01/vocabulary/Product97476	befogs
http://www4.wiwiss.fu-berlin.de/bizer/bsbm/v01/vocabulary/Product180800	directs
http://www4.wiwiss.fu-berlin.de/bizer/bsbm/v01/vocabulary/Product156288	overhauls
http://www4.wiwiss.fu-berlin.de/bizer/bsbm/v01/vocabulary/Product99042	vesper's
http://www4.wiwiss.fu-berlin.de/bizer/bsbm/v01/vocabulary/Product97088	witness's

Diagnosis:

• There is always one line that differs
• The value of last line change if re-run.
• This bug happens rarely and randomly: only happened once out of 49 / 1491 steps.

[mhoangvslev]

Hypotheses:

• [ ] cache problems (again). Simple re-execute the rule.
• [x] The LIMIT keyword limits the number of returned solution mappings. Since the results are returned in a different order, the odd line might have come from a different source.

[mhoangvslev]

In snakefile, I specify the retries: 1 and the problem doesn't re-appear for this q05 of batch 0. Maybe next batch?

[mhoangvslev]

• [x] Re-activate ORDER BY might resolve the issue.

[mhoangvslev]

Seems okay so far...

[mhoangvslev]

For it to work, @momo54 suggested using ORDER BY with every projected variable. It works.