syphax-bouazzouni
commented
11 months ago In the section, I will run the same query in the different code bases and give the generated SPARQL query
The query will be the following
University.where.include(:name, programs: [:name]).allNCBO code
# Get the universities including their name and programs (inverse of university property)
SELECT DISTINCT ?id ?name ?programs
FROM <http://goo.org/default/University>
FROM <http://goo.org/default/Program>
WHERE {
?id a <http://goo.org/default/University> .
OPTIONAL { ?id <http://goo.org/default/name> ?name . }
OPTIONAL { ?programs <http://goo.org/default/university> ?id . }
}
# Get the 9 programs names
SELECT DISTINCT ?id ?name
FROM <http://goo.org/default/Program>
WHERE {
?id a <http://goo.org/default/Program> .
OPTIONAL { ?id <http://goo.org/default/name> ?name . }
FILTER(?id = <http://example.org/program/UPM/CompSci> || ?id = <http://example.org/program/UPM/BioInformatics> || ?id = <http://example.org/program/UPM/Medicine> || ?id = <http://example.org/program/Stanford/Medicine> || ?id = <http://example.org/program/Stanford/BioInformatics> || ?id = <http://example.org/program/Stanford/CompSci> || ?id = <http://example.org/program/Southampton/CompSci> || ?id = <http://example.org/program/Southampton/Medicine> || ?id = <http://example.org/program/Southampton/BioInformatics>)
}Notes:
- Simple
- Dynamic select clause, depending on he properties to include.
- Use an optional for each of the properties to include (e.g name)
- Good for fetching elements that don't have a lot of attributes; but not for elements that have a lot of attributes (e.g submission metadata in our case)
LIRMM code for 4store and GraphDb
# Get the universities including their name and programs (inverse of university property)
SELECT DISTINCT ?id ?attributeProperty ?attributeObject
FROM <http://goo.org/default/University>
FROM <http://goo.org/default/Program>
WHERE {
?id a <http://goo.org/default/University> .
OPTIONAL {
{ ?id <http://goo.org/default/name> ?attributeObject . BIND( "name" as ?attributeProperty) }
UNION
{ ?attributeObject <http://goo.org/default/university> ?id . BIND( "programs" as ?attributeProperty) }
}
}
# Get the 9 programs names
SELECT DISTINCT ?id ?attributeProperty ?attributeObject
FROM <http://goo.org/default/Program>
WHERE {
?id a <http://goo.org/default/Program> .
OPTIONAL {
{ ?id <http://goo.org/default/name> ?attributeObject . BIND( "name" as ?attributeProperty) }
}
FILTER(?id = <http://example.org/program/UPM/CompSci> || ?id = <http://example.org/program/UPM/BioInformatics> || ?id = <http://example.org/program/UPM/Medicine> || ?id = <http://example.org/program/Stanford/Medicine> || ?id = <http://example.org/program/Stanford/BioInformatics> || ?id = <http://example.org/program/Stanford/CompSci> || ?id = <http://example.org/program/Southampton/CompSci> || ?id = <http://example.org/program/Southampton/Medicine> || ?id = <http://example.org/program/Southampton/BioInformatics>)
}Notes
- Complex
- Fixed select always
SELECT DISTINCT ?id ?attributeProperty ?attributeObject, making it more performant for large elements with a lot of metadata (submission in our case) - Use a UNION for each of the properties to include (e.g name), required to have the fixed select clause.
- Slightly worst performance on average.
LIRMM code for Virtuoso and Allgeropgraph
# Get the universities including their name and programs (inverse of university property)
SELECT DISTINCT ?id ?attributeProperty ?attributeObject
FROM <http://goo.org/default/University>
FROM <http://goo.org/default/Program>
WHERE {
?id a <http://goo.org/default/University> .
OPTIONAL {
{ ?id ?attributeProperty ?attributeObject . FILTER(?attributeProperty = <http://goo.org/default/name>) }
UNION
{ ?attributeObject ?attributeProperty ?id . FILTER(?attributeProperty = <http://goo.org/default/university>) }
}
}
# Get the 9 programs names
SELECT DISTINCT ?id ?attributeProperty ?attributeObject
FROM <http://goo.org/default/Program>
WHERE {
?id a <http://goo.org/default/Program> .
OPTIONAL {
{ ?id ?attributeProperty ?attributeObject . FILTER(?attributeProperty = <http://goo.org/default/name>) }
}
FILTER(?id = <http://example.org/program/Stanford/BioInformatics> || ?id = <http://example.org/program/Stanford/CompSci> || ?id = <http://example.org/program/Stanford/Medicine> || ?id = <http://example.org/program/Southampton/BioInformatics> || ?id = <http://example.org/program/Southampton/CompSci> || ?id = <http://example.org/program/Southampton/Medicine> || ?id = <http://example.org/program/UPM/BioInformatics> || ?id = <http://example.org/program/UPM/CompSci> || ?id = <http://example.org/program/UPM/Medicine>) }Notes
- A no-complex query, static select clause, static where clause, not dependent on the complexity of the query. The form will be the same only the filters change depending on the properties to include.
- Does not work in Graph DB and 4store
jonquet
codecov[bot]
Prerequisites
Context
This PR makes the ths following triple stores:
4store(fs),GraphDb(gb),Virtuoso(vo), andAllegroGraph(ag), work with Goo and add their test workflows.The challenge was that the same query was not parsed from one triple store to another. i.e. the same SPARQL query returns different things. form store to another.
So
GraphDbworks the same as4store, and is integrated easily into the current code and the way we generate our queries (see an example of a generated query in the section "Generated Queries examples ").VirtuosoandAllegroGraphsupport a better way of handling queries, so we updated our code to write better and simpler queries (see an example of a generated query in the section "Generated Queries examples ").Benchmarks
The table below shows the time that it takes to run all the tests in each of the triple-store, with different slice configurations (slices, is a configuration variable to define how we slice our fetched data, e.g. if we query 100 elements and the slice is 20 we will do 5 requests fetching 20 elements each)
The benchmarks up use our version code not the NCBO one, and we use a different approach to building queries since https://github.com/ncbo/goo/pull/124, so below the benchmark using the NCBO code for 4store and AllegroGraph
So to summarize here is the final board
Generated Queries examples
See https://github.com/ontoportal-lirmm/goo/pull/48#issuecomment-1763725927
Changes