search

Swirrl / drafter

A clojure service and a client to it for exposing data management operations to PMD
Other
0 stars 0 forks source link

Move state graph into separate database to allow uncontended state updates #163

Open RickMoynihan opened 7 years ago

RickMoynihan commented 7 years ago

Stardog currently can't guarantee that any write to the state graph won't block behind a large copy operation. This is a problem in a post #155 world where :batch-write operations can take 15-20 minutes, as it means the draftset UI can block indefinitely on requests to create/edit/share draftsets.

By moving the state graph in to a separate secret database, we can remove the contention on writes and ensure that large graph operations don't cause small state updates to block indefinitely.

The problem with this is that it means updates to state and graphs will no longer be transactional, however the window for error is very small and if an error does occur you'll still be in a recoverable state. So it's still worth doing this despite the lack of transactionality.

Regardless of this, we should also implement a more robust transaction mechanism where by we write a change id into the state database, perform the publish, then write the chage-id into the data database. Making this better is documented in issue #164

ricroberts commented 7 years ago

This is less important now we're going to make sync writes check/attain the exclusive lock. i.e we'll fail early and hard if the sync write isn't going to happen quickly.

ricroberts commented 1 year ago

I'm not sure if this still applies on later versions of stardogs either!