search

fluree / db

Fluree database library
https://fluree.github.io/db/
Other
330 stars 21 forks source link

Shacl optimization #792

Closed zonotope closed 1 month ago

zonotope commented 1 month ago

This patch ensures that we run a query for shacl shapes present in a db only once while validating the set of modified subjects in a transaction, and that we bypass shacl all together if there are no shapes to validate.

There's also an optimization for new index jobs processed in the index queue to always use the latest known index before processing the new indexing job. This should prevent the indexer from repeating work it has already done.

this patch depends on #785, so please review that one first.

zonotope commented 1 month ago

If I'm understanding correctly, these are the optimizations:

For SHACL, we only look up the shape sids once, instead for every single subject in the transaction.

Also, when processing a db from index queue we'll make sure to assoc in the most recent index, instead of whatever the index was when the db was put on the indexing queue.

Not quite for the second one. We don't assoc in the recent index because that wouldn't actually build the index in the db. We reload the db with the recent index so both the index roots and novelty are correct.