jakubklimek
commented
7 years ago Specifically, http://purl.org/dc/terms/title is not the same as http://purl.org/dc/elements/1.1/title, although they have the same meaning and in Linked Data terms should be interchangeable.
dc: elements are deprecated in favor of dcterms: for usage in Linked Data. I don't see it used in the query, is it used in some specific datasets? Then it should be changed to dcterms:
Is there a way to run SPARQL queries in such a liberal way that takes into account all the alternative possibilities in a performant way?
Sometimes it is possible to rewrite OPTIONALs into UNIONs when there are not many, which is often faster to execute.
The same goes also for the fiscalYear/fiscalPeriod properties and for their values which contain the "gregorian" part, while the other datasets don't. All these differences pose a great risk both to performance and to the correctness of the queries built by rudolf.
On the other hand, this kind of heterogenity is a quite natural aspect of Linked Data. In addition, the specific string of the IRI should not matter, as it is not to be parsed anyway. If, for example, a label for the year or period is needed, the IRI should be dereferenced and the label taken from the data. For performance reasons, this can be done in advance by an enhancement pipeline which would cache these responses or their parts in a separate named graph.
In general 1) I don't see the point in querying over all the heterogeneous datasets at once using one query - what sort of question this query answers? A more typical workflow would be for the user to select a few datasets with which he/she wants to work, I suppose. Do we have any user stories to support one or the other? 2) It is natural to have some level of inconsistency. Instead of trying to solve them all in one query, this can be resolved progressively e.g. by normalizing pipelines, which compute the missing properties if possible (e.g. one specific label (e.g. dcterms:title) out of all used options(e.g. dcterms:title, schema:name, foaf:name, skos:prefLabel, ...), fiscalPeriod from fiscalYear, etc.) so that so many OPTIONALs do not have to be used.
larjohn
jindrichmynarz
schmaluk
pwalsh
Hello @larjohn, @jindrichmynarz , @jakubklimek and @marek-dudas , I thought it would be better to have this on github rather than in email correspondance for better collaboration. So I have put the mail correspondance between @larjohn and me now here. (I hope I have understood @larjohn correctly and trying to repeat it.)
Remark: @marek-dudas This problem occurred when you search in the http://eis-openbudgets.iais.fraunhofer.de/explorer for the datasets which have been imported by the FDP-to-RDF-pipeline. They arent shown in the Explorer.
There seems to be some differences in the naming of properties between different datasets which might not affect their correctness but might have a big effect on constructing performant SPARQL-Queries:
Please see the following SPARQL-Query in Rudolf:
All these OPTIONAL are lowering the performance. If all the datasets do have these attributes, the optionals can be removed and a lot of performance can be gained.
Is there a way to run SPARQL queries in such a liberal way that takes into account all the alternative possibilities in a performant way?
Can we somehow unify the naming of the properties & their values in the datasets? The validation & clearance pipeline would be still missing, which maybe could be used to improve the quality of the datasets in that regard and afterwards move datasets from the staging DB to the production DB. Or would another solution be better and easier here?
Thank you for your help