Implement catch-up storage and use transactions for `Delivery` routines

This changeset brings the support of catch-up operations to the applications running on top of Google Datastore.

DsCatchUpStorage

A newDsCatchUpStorage is now responsible for storing the state of the catch-up processes.

Please note, in its work, this implementation is not using Datastore transactions. As the framework has no goal to handle the concurrent access to the state of a particular catch-up process, the code has been left fairly simple.

Modes for DsInboxStorage

DsInboxStorage has been modified to support transactional operations when operating on top of Firestore in Datastore mode.

In particular, in this mode, the InboxMessage records are stored as children of ShardIndex. That allows querying and modifying them in the scope of a Datastore transaction. For a framework user, it means that all operations performed during the message delivery and catch-up are executed strictly in the desired order, as Datastore provides serializable isolation for its transactions.

Legacy native Datastore operational mode is also supported, but due to its limits and risk of contention on record operations, no transactional work is performed.

By default, the DatastoreStorageFactory is instantiated with the DsInboxStorage in native mode. It allows running the unit tests on the Datastore emulator, which so far only acts like a Datastore in native mode.

To switch to the Firestore in Datastore mode, an SPI user should override the DatastoreStorageFactory.inboxStorageWith(multitenant, datastore) method and return the DsInboxStorage created with DatastoreMode.FIRESTORE_AS_DATASTORE value:

    @Override
    protected InboxStorage inboxStorageWith(DatastoreWrapper wrapper,
                                            boolean multitenant) {
        return new DsInboxStorage(wrapper, 
                                  multitenant,
                                  DatastoreMode.FIRESTORE_AS_DATASTORE);
    }

Please note that other storage implementations so far do not require the specific support of Datastore operational mode. Thus, their code has remained intact.

Improved shard pick-up

The DsShardedWorkRegistry has been updated to perform the shard pick-up in the scope of a Datastore transaction. It allows dealing with the potential concurrency issues in a more natural platform-specific way, rather than trying to read-write-read the same record as previously.

This feature is intended to work notwithstanding the Datastore mode.

Other changes

The API of TransactionalWrapper and DsMessageStorage was updated to allow handling the parent-children (ancestor) relations for the stored records.

The sample Datastore index files were updated according to the latest changes. There is also a new sample index set for Firestore in Datastore mode. For more details, see /datastore/config/README.md.

The library version is set to 1.4.11.

SpineEventEngine / gcloud-java

Implement catch-up storage and use transactions for `Delivery` routines #146

Codecov Report