Concept: Hooking into the command/event flow // Extensibility

[skurfuerst]

Concept for hooking into the Command / Event Flow

(Discussion @bwaidelich, @kitsunet , @skurfuerst )

Use Cases

• Neos UI needs to remember to which Document a deleted node belongs; so that it can publish them.
• We want to build up a "Derived event stream" which per-document-node shows e.g. "the URL of this page has changed"
• (maybe): Neos UI needs to remember how many changes exist for the current page.

Proposal

• Introduce a Command Bus in the EventSourcing package
• command handlers become internal (non-public).
• all commands will get sent to the command bus; and the command handlers register themselves at the command bus.
• The command bus works in a "Middleware style", i.e. is a sorted chain of handlers (e.g. with PositionalArraySorter).
• (optional) we might think about separating the validation step from the "doing step"; as it is currently interleaved in a single method in the CommandHandler.

Implications

We see the Neos UI in the example below as separate application

e.g. for the Neos UI to remember to which document a deleted node belongs, the Neos UI would hook into the following Commands:

• DeleteNode
• PublishNode

On DeleteNode, it will read the projection and remember the parent document node. Remembering in this case can mean directly storing it in the database; or working as a separate event-sourced application (with optionally a separate event store as well).

On PublishNode, we will copy the "Deleted Node" lines from our custom storage table, to the target ContentStream (if the target is not the live WS).

This leaves us with a small inconsistency window, as the separate application has different consistency guarantees. However, for our cases this is very very likely not a problem.

Alternatives (not suggested)

• Hook into the projector before processing an event -- very dangerous, we do not want this as an explicit extension point
• a removedContentShown flag in the CR (similar to Enable/Disable) -- after quite some discussions, we believe this is not a core CR concern.

[bwaidelich]

Related to: https://github.com/neos/Neos.EventSourcing/issues/118

[bwaidelich]

After letting the proposal sink in for a few days it still feels a little awkward to access a "foreign" (as in: different Bounded Context) Read Model from the Write Side. But it might be a very pragmatic solution that just works out in practice – we'll have to give it a try. I'll introduce an external Command Bus (league/tactician) to the CR to see if this works.

Nevertheless I want to add some more potential alternatives for the record:

• Add identifiers of deleted nodes to the payload of the NodeAggregateWasRemoved event (probably not viable because there could be thousands of them)
• Publish "derived" events of all removed nodes (possibly to separate streams) – we already consider this for the live workspace (see #21) but maybe it's an option for other workspaces, too
• Turn the Content Graph into a Persistent Data Structure, that could be queried for things like: removed nodes between ContentStream X @ Version Y and ContentStream X' @ Version Y' (quite interesting IMO but that would mean a major change in the inner workings of the Graph and it might just be too complex & slow in practice)
• Add a (simplified) version of the Graph to the Write Side of the CR that acts like an Aggregate¹ that is only concerned about the node edges (that would not solve the issue immediately but it somehow feels "cleaner" to ask that Write Model at command time instead of some projected Read Model)

---

¹ It's probably not feasible to have an actual Aggregate that reconstitutes itself in memory – instead it could always materialize it's state in some database as soon as new events come in

[bwaidelich]

See first draft: https://github.com/neos/contentrepository-development-collection/pull/104 (I moved the longer comment to the PR)

[bwaidelich]

After our last ESCR talk I think that we need something other than a (global) Command Bus. Instead it should (IMO) be an instance of the "Content Repository" that will act as single authority for...

Dispatching Commands

In that sense it is a Command Bus, but not a global one. That way we could handle commands for different CR instances "at the same time". Besides, the API will be easier to use because the correct CommandHandler instance does not need to be known and it could be a single public method, like:

$neosCR->handle(new CreateNodeVariant(...));

Interacting with Core Read Models

It is to be defined, what the "Core Read Models" are, but I think it should be the Content(Sub)Graph and some kind of Diff between two content streams (that also considers deleted nodes)

Registering extensions

We need something more specific than (global) event listeners. Instead I could think of two kinds of extensions:

1. low level: A special CR listener that will receive all events of a given CR event store
2. high level: A callback that is invoked whenever the ContentGraph of a given CR is updated (this will allow for interaction with the just-updated graph, e.g. for hierarchical queries)

Some more thoughts

• CR instances probably need to be configured explicitly (so they can have individual dimension and event store configurations etc)
• Neos should provide a factory that determines the "default CR instance" based on the current request (via middleware)
• We can use the instances explicitly in Behat tests (e.g. in order to test multiple Content Graph implementations in one setup)
• This API will allow us to introduce the Content Stream Aggregate (#195) without breaking the interface
• The CR instance could know the "current initiating user" and augment the events accordingly (see #196)