laura-slaughter commented 8 years ago

I am trying to run several queries to a large repository (drugbank) in dydra. I get a server timeout and I am wondering if it is possible to set a larger timeout.

SELECT DISTINCT ?prop FROM http://bio2rdf.org/drugbank_resource:bio2rdf.dataset.drugbank.R4 WHERE { ?obj a http://bio2rdf.org/drugbank_vocabulary:Half-life . ?obj ?prop ?targetObj . FILTER (isLiteral(?targetObj)) }

SELECT (DATATYPE(?targetObj) as ?type) FROM http://bio2rdf.org/drugbank_resource:bio2rdf.dataset.drugbank.R4 WHERE { ?obj a http://bio2rdf.org/drugbank_vocabulary:Half-life . ?obj http://www.w3.org/2000/01/rdf-schema#label ?targetObj . } LIMIT 1

Server response: "

504 Gateway Time-out

504 Gateway Time-out

nginx/1.4.6 (Ubuntu)

lisp commented 8 years ago

while the repository is beyond the customary size for accounts on that host, it is not very large. it appears that the compiler makes an unfortunate choice as to how to combine the filter with the join. this causes the filter rate to drop well below the normal +/-1M/second and the query times out.

i will investigate, but, in the meantime, i saved on your view page several variations which avoid the issue.

lisp commented 8 years ago

there are three issues involved in this:

as a rule, filters which can be applied to an elementary pattern are pushed into the bgp processing step. in the case of your queries, the consequence is a filtered scan of the full repository content - even though the filter would be more judiciously applied to the constrained solutions which result from the join. in general that requires comparing the effort and compound cardinality of the joined expression with that of the direct filter. this is not presently done, but suggests, as a rule, it is a bad idea to push a filter onto a full scan. we will pursue this.
wherever a filter is supplied, an effort is made to require only that information from the repository which is required to apply the predicate. this was effective in the case of the isolated filter, which contributed to its shorter execution. this has now been corrected for the inlined case.
sparql filter semantics defines exceptions as a false value. this entails more complex handling in the inlined case than when the filter is isolated. as this aspect is not nearly as significant as the second one, we will not pursue it for the time being.

the most significant aspect, limiting the information required to apply the predicate, has been addressed, with the consequence that the original query now completes within the standard time constraint.

dydra / support

server timeout #44

504 Gateway Time-out