ICOS-Carbon-Portal / meta

ICOS Carbon Portal metadata service
GNU General Public License v3.0
3 stars 1 forks source link

[sparql] Asking for cmpeta:hasSha256sum makes an otherwise fast query slow #210

Closed mirzov closed 1 year ago

mirzov commented 1 year ago

The query:

prefix cpmeta: <http://meta.icos-cp.eu/ontologies/cpmeta/>
prefix prov: <http://www.w3.org/ns/prov#>
select * where{
    ?dobj cpmeta:hasObjectSpec ?specUri;
        cpmeta:wasSubmittedBy/prov:endedAtTime ?submTime;
        cpmeta:hasSha256sum ?hash
}
order by ?submTime
limit 10
offset 250000

Asking for file name instead of the hashsum is fast, and used to be fast even when the file names were stored in the database only (not in the "magic" index).

mirzov commented 1 year ago

After an extra investigation it is clear that the culprit is the offset. It's impossible to "magically" optimize the offset when a non-magic basic statement pattern appears in the query (one cannot know how many of the data objects actually do have the property in question). So, the offset and limit are necessarily non-magical in this case.