ad-freiburg / qlever

Very fast SPARQL Engine, which can handle very large knowledge graphs like the complete Wikidata, offers context-sensitive autocompletion for SPARQL queries, and allows combination with text search. It's faster than engines like Blazegraph or Virtuoso, especially for queries involving large result sets.
Apache License 2.0
424 stars 52 forks source link

Fix out of memory issue with transitive paths #1627

Closed RobinTF closed 2 days ago

RobinTF commented 3 days ago

The introduction of lazy transitive paths via #1595 caused transitive path operations with lazy inputs and a fully materialized result to store the same local vocab once per input row. This lead to OOM errors even if that local vocab was empty. This commit fixes that behavior by

  1. Not merging empty local vocabs at all
  2. Only storing the local vocab for each input block, without repeating them for each row.

Note that this still might lead to redundant stores in the local vocab, but deduplicating those will be done in a separate PR.

sparql-conformance[bot] commented 3 days ago

Conformance check passed ✅

No test result changes.

Details: https://qlever.cs.uni-freiburg.de/sparql-conformance-ui?cur=cb06fa38b1eb261009c4412bc3d54bdbfea91268&prev=55e0617ba75182958fc16579eaad6731b4a54ea2

codecov[bot] commented 3 days ago

Codecov Report

Attention: Patch coverage is 92.30769% with 1 line in your changes missing coverage. Please review.

Project coverage is 89.28%. Comparing base (39ca684) to head (cb06fa3). Report is 4 commits behind head on master.

Files with missing lines Patch % Lines
src/engine/TransitivePathImpl.h 87.50% 0 Missing and 1 partial :warning:
Additional details and impacted files ```diff @@ Coverage Diff @@ ## master #1627 +/- ## ========================================== - Coverage 89.30% 89.28% -0.03% ========================================== Files 373 374 +1 Lines 35377 35610 +233 Branches 3999 4020 +21 ========================================== + Hits 31594 31794 +200 - Misses 2493 2520 +27 - Partials 1290 1296 +6 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


🚨 Try these New Features:

sonarcloud[bot] commented 2 days ago

Quality Gate Passed Quality Gate passed

Issues
1 New issue
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud