ad-freiburg / qlever

Very fast SPARQL Engine, which can handle very large knowledge graphs like the complete Wikidata, offers context-sensitive autocompletion for SPARQL queries, and allows combination with text search. It's faster than engines like Blazegraph or Virtuoso, especially for queries involving large result sets.
Apache License 2.0
424 stars 52 forks source link

Add `DeltaTriplesManager` #1603

Closed joka921 closed 2 weeks ago

joka921 commented 2 weeks ago

The already existing DeltaTriples class maintains a dynamically changing set of insertions and deletions relative to the original input data, together with a (single) local vocab. The class is not threadsafe and has to be used with care. In particular, concurrent update queries have to be serialized, and while a query makes use of the "delta triples", it has to be made sure that they are not changed over the course of the processing of that query.

Both of these problems are solved by the new DeltaTriplesManager class. The index has a single object of this class. It maintains a single DeltaTriples object, write access to which is strictly serialized. Each new query gets a so-called snapshot of the current delta triples. This is a full copy (of the delta triples located in each of the permutations and of the local vocab). These snapshots are read-only and multiple queries can share the same snapshot. A snapshot lives as long as one query using it is still being processed.

codecov[bot] commented 2 weeks ago

Codecov Report

Attention: Patch coverage is 89.06250% with 14 lines in your changes missing coverage. Please review.

Project coverage is 89.16%. Comparing base (bb70c4a) to head (d70705b). Report is 1 commits behind head on master.

Files with missing lines Patch % Lines
src/index/Index.cpp 68.75% 5 Missing :warning:
src/engine/CountAvailablePredicates.cpp 0.00% 4 Missing :warning:
src/index/IndexImpl.h 25.00% 3 Missing :warning:
src/index/IndexImpl.cpp 84.61% 2 Missing :warning:
Additional details and impacted files ```diff @@ Coverage Diff @@ ## master #1603 +/- ## ========================================== - Coverage 89.17% 89.16% -0.01% ========================================== Files 372 372 Lines 34579 34635 +56 Branches 3912 3912 ========================================== + Hits 30835 30883 +48 - Misses 2471 2479 +8 Partials 1273 1273 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

Qup42 commented 2 weeks ago

Some SPARQL operations may result in Quad insertions and deletions. Is it possible that operations acquire the lock in between these two parts of a single update?

sparql-conformance[bot] commented 2 weeks ago

Conformance check passed ✅

No test result changes.

Details: https://qlever.cs.uni-freiburg.de/sparql-conformance-ui?cur=d70705b75854bcf5d42885e2ca05a8a64f7e828d&prev=bb70c4a8bd9bb9505ba809ba845b0bd7df0dc9b6

sonarcloud[bot] commented 2 weeks ago

Quality Gate Passed Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarCloud

joka921 commented 2 weeks ago

@hannahbast I have also looked at the lines, but they are either completely unrelated to this PR, or they will be adressed anyway in one of the follow-up PRs.