The goal of this issue is to enable the possibility to do a couple of things:
Easily add events to the outbox of an event store from the client through a specific interface for it (IEventOutbox)
Be able to easily set up subscriptions from one event store into another with a specific inbox for the source (Do not mix events from multiple in one)
Add a clear capability for Reactors, Reducers and Projections to specify the inbox for the event store they want as source, without having to know how Chronicle names the inbox event sequence per event store.
Process should run completely server side (Kernel), allowing for it to run without a client connected.
.NET Client
[ ] Add an interface called IEventOutbox in the EventSequences namespace, similar to what we have for IEventLog
[ ] Expose the IEventOutbox as Outbox property in the IEventStore
[ ] Add optional inboxFor parameter to ReactorAttribute, leveraging the EventSequenceId.InboxPrefix to create correct the EventSequenceId to use as source. The parameters eventSequence and inboxFor should be mutual exclusive. Throw specific exception from the attributes constructor if both are specified (e.g. AmbiguousEventSequenceSpecifiedForReactor).
[ ] Add optional inboxFor parameter to ReducerAttribute, leveraging the EventSequenceId.InboxPrefix to create correct the EventSequenceId to use as source. The parameters eventSequence and inboxFor should be mutual exclusive. Throw specific exception from the attributes constructor if both are specified (e.g. AmbiguousEventSequenceSpecifiedForReducer).
[ ] Add ability from client to instruct Kernel to setup subscription from the outbox of one EventStore to be put into a specific inbox. Typically we'd like to have a simple method on the IEventStore API: Task SubscribeToOutbox(EventStoreName).
ASP.NET Client
[ ] On the ChronicleAspNetCoreOptions we should have a nice way to configure any event stores we're interested in. A List of EventStoreNames (e.g. EventStoreOutboxSubscriptions). This would then automatically hook this up in the ChronicleClientStartupTask after discovery and registering.
[ ] Add registration of IEventOutbox in the ServiceCollection
Orleans In-Process Client
[ ] Hook up bindings for circumventing the gRPC, since we're in process
[ ] On the ChronicleOrleansInProcessOptions we should have a nice way to configure any event stores we're interested in. A List of EventStoreNames (e.g. EventStoreOutboxSubscriptions). This would then automatically hook this up in the ChronicleClientStartupTask after discovery and registering in the Orleans.InProcess project.
[ ] Add registration of IEventOutbox in the ServiceCollection
Kernel
Observer type
We need an observer type for the system to be able to do the actual copying of events from an outbox to an inbox.
This observer type needs to be able to run even if a client is disconnected, meaning that it will never send anything to the client.
The kernel side we implement to be extensible and open for other use cases other than just the Outbox/Inbox pattern.
[ ] Add a new observer type called EventSequenceForwarder in a new namespace in Observation namespace in the Grains project called EventSequences.
[ ] Implement a grain called EventSequenceForwarder, its job is to setup the subscription from a definition. The definition (EventSequenceForwarderDefinitionwill typically contain information about the sourceEventStoreNamethe sourceEventSequenceIdand the targetEventStoreNameand the targetEventSequenceId`
[ ] Implement an ObserverSubscriber - typically named EventSequenceForwarderObserverSubscriber. Look at how Reactors or Reducers are implemented.
[ ] Implement an EventSequenceForwarderManager similar to the ProjectionsManager that is responsible for automatically start all forwarders on startup. Activated from the ChronicleServerStartupTask. It should also be responsible for automatically adding forwarders when a new namespace gets added.
Storage
[ ] New storage interface similar to Reactors / Reducers for storing and retrieving definitions for the EventSequenceForwarder. This should be on the IEventStoreStorage interface.
[ ] MongoDB implementation of the IEventStoreStorage interface.
[ ] Implement a EventSequenceForwarderDefinitionStorageProvider, similar to ReactorDefinitionStorageProvider
[ ] Register storage provider in StorageProviderExtensions in theSetup` project.
gRPC Contract
[ ] Extend the IEventStores contract equivalent to the SubscribeToOutbox() method that takes a request with the target EventStoreName (The EventStore we're asking to set up the subscription for in the client) and source EventStoreName
gRPC Services
[ ] The implementation of IEventStores should use the EventSequenceForwarder to set up the subscription. Similar to how Projections service work.
einari
commented
1 month ago
The goal of this issue is to enable the possibility to do a couple of things:
IEventOutbox).NET Client
IEventOutboxin the EventSequences namespace, similar to what we have forIEventLogIEventOutboxasOutboxproperty in theIEventStoreinboxForparameter toReactorAttribute, leveraging theEventSequenceId.InboxPrefixto create correct theEventSequenceIdto use as source. The parameterseventSequenceandinboxForshould be mutual exclusive. Throw specific exception from the attributes constructor if both are specified (e.g. AmbiguousEventSequenceSpecifiedForReactor).inboxForparameter toReducerAttribute, leveraging theEventSequenceId.InboxPrefixto create correct theEventSequenceIdto use as source. The parameterseventSequenceandinboxForshould be mutual exclusive. Throw specific exception from the attributes constructor if both are specified (e.g. AmbiguousEventSequenceSpecifiedForReducer).EventStoreto be put into a specific inbox. Typically we'd like to have a simple method on theIEventStoreAPI:Task SubscribeToOutbox(EventStoreName).ASP.NET Client
ChronicleAspNetCoreOptionswe should have a nice way to configure any event stores we're interested in. A List of EventStoreNames (e.g. EventStoreOutboxSubscriptions). This would then automatically hook this up in theChronicleClientStartupTaskafter discovery and registering.IEventOutboxin theServiceCollectionOrleans In-Process Client
ChronicleOrleansInProcessOptionswe should have a nice way to configure any event stores we're interested in. A List of EventStoreNames (e.g. EventStoreOutboxSubscriptions). This would then automatically hook this up in theChronicleClientStartupTaskafter discovery and registering in theOrleans.InProcessproject.IEventOutboxin theServiceCollectionKernel
Observer type
We need an observer type for the system to be able to do the actual copying of events from an outbox to an inbox. This observer type needs to be able to run even if a client is disconnected, meaning that it will never send anything to the client.
The kernel side we implement to be extensible and open for other use cases other than just the Outbox/Inbox pattern.
EventSequenceForwarderin a new namespace inObservationnamespace in theGrainsproject calledEventSequences.EventSequenceForwarder, its job is to setup the subscription from a definition. The definition (EventSequenceForwarderDefinitionwill typically contain information about the sourceEventStoreNamethe sourceEventSequenceIdand the targetEventStoreNameand the targetEventSequenceId`ObserverSubscriber- typically namedEventSequenceForwarderObserverSubscriber. Look at how Reactors or Reducers are implemented.EventSequenceForwarderManagersimilar to theProjectionsManagerthat is responsible for automatically start all forwarders on startup. Activated from theChronicleServerStartupTask. It should also be responsible for automatically adding forwarders when a new namespace gets added.Storage
EventSequenceForwarder. This should be on theIEventStoreStorageinterface.IEventStoreStorageinterface.EventSequenceForwarderDefinitionStorageProvider, similar toReactorDefinitionStorageProviderStorageProviderExtensions in theSetup` project.gRPC Contract
IEventStorescontract equivalent to theSubscribeToOutbox()method that takes a request with the targetEventStoreName(The EventStore we're asking to set up the subscription for in the client) and sourceEventStoreNamegRPC Services
IEventStoresshould use theEventSequenceForwarderto set up the subscription. Similar to howProjectionsservice work.