This patch schedules indexing jobs for new commits with a dropping queue to ensure we are always working on the latest commit. It also separates commit updates to ledger/branch state from index updates to simplify and streamline those updates, and all of these updates and decisions about when/if to make them are executed entirely within an atomic swap! to branch state to eliminate race conditions.
In order to enable atomic updates to index state, new dbs are loaded in the background within an AsyncDB. To enable interoperability between AsyncDB and JsonLdDb, I had to add more protocols for history queries, policy restriction, and time travel.
I also added a new indexing protocol for db record types that replaces the standalone indexer records and iIndex protocols. With this patch, all index and commit updates to ledger state are atomic and without any locks.
There is still a lot of cleanup and reorganization to do, but this patch is the minimal changeset necessary to manage indexing jobs with a queue.
This patch schedules indexing jobs for new commits with a dropping queue to ensure we are always working on the latest commit. It also separates commit updates to ledger/branch state from index updates to simplify and streamline those updates, and all of these updates and decisions about when/if to make them are executed entirely within an atomic
swap!to branch state to eliminate race conditions.In order to enable atomic updates to index state, new dbs are loaded in the background within an
AsyncDB. To enable interoperability betweenAsyncDBandJsonLdDb, I had to add more protocols for history queries, policy restriction, and time travel.I also added a new indexing protocol for db record types that replaces the standalone indexer records and
iIndexprotocols. With this patch, all index and commit updates to ledger state are atomic and without any locks.There is still a lot of cleanup and reorganization to do, but this patch is the minimal changeset necessary to manage indexing jobs with a queue.