9.0: Correctly set Correlation and Causation IDs

[skurfuerst]

• from 1 Command, multiple Events occur (Create Node with tethered child nodes)
  • Causation id (seems to be already impl.; not yet tested)
• Publishing of nodes

  • Correlation ID for full publish process for ALL involved events.

Causation ID == "Command ID" (we don't store it, just conceptually)

---

PROBLEMS / concepts:

• Publishing (applying events to other stream)
• Rebase (applying COMMANDS to other stream) - maybe later "applying events to other stream, but they can fail"
• Node with tethered children (multiple events from one command)

Intuition:

• Causation ID should not change once set.

[bwaidelich]

Our current idea, as discussed:

• initiatingUserId, initiatingTimestamp and causationId are copied from initial events (Note: currently from the first event of a batch, which has the command attached to it in the metadata))
• correlationId is always set/overridden on publishing/rebasing (= workspace related operations)

Example

Authenticated User "user1"

Create Node "home"

with tethered node "home/main"

• NodeAggregateWasCreated

  • stream: "cs-user1-a"
  • id: "e1"
  • initiatingUserId: "user1"
  • initiatingTimestamp "t1"
  • causationId "causationId1" (generated) (TODAY: not set)
  • correlationId: ~

• NodeAggregateWasCreated "home/main" (tethered)

  • stream: "cs-user1-a"
  • id "e2"
  • initiatingUserId "user1"
  • initiatingTimestamp "t1'"
  • causationId "causationId1" (generated) (TODAY: "e1")
  • correlationId: ~

Authenticated User "user2"

Create node "service"

• NodeAggregateWasCreated "service"
  • stream: "cs-user2-a"
  • id "e3"
  • initiatingUserId "user2"
  • initiatingTimestamp "t2"
  • causationId "causationId2" (generated)
  • correlationId: ~

Authenticated User "user3"

Publishing of "user2" changes

just imagined, this is not really possible – but it's relevant for review workspaces

• no rebase necessary (TODAY: we always rebase(?))

• => copying of events from "cs-user2-a" to "cs-live"

• NodeAggregateWasCreated "service"

  • stream: "cs-live"
  • id "e4"
  • initiatingUserId "user2" (copied from the original event "e3")
  • initiatingTimestamp "t2" (copied from the original event "e3")
  • causationId "causationId2" (copied from the original event "e3")
  • correlationId: "correlationId1"

• WorkspaceWasPublished

  • stream: "workspace-live"
  • id: "e5"
  • initiatingUserId "user3"
  • initiatingTimestamp "t3"
  • causationId (?)
  • correlationId: "correlationId1" (generated for this publishing process)

Authenticated User "user1"

Rebasing

• From "cs-user1-a" to "cs-user1-b" (because base content stream ("live") has new events)

• TODAY: Re-handle commands, future hopefully: Re-apply events (can fail)

• NodeAggregateWasCreated "home"

  • stream: "cs-user1-b"
  • id "e6"
  • initiatingUserId "user1" (copied from event "e1")
  • initiatingTimestamp "t1" (copied from event "e1")
  • causationId "causationId1" (copied from event "e1")
  • correlationId: "correlationId2"

• NodeAggregateWasCreated "home/main" (tethered)

  • stream: "cs-user1-a"
  • id "e7"
  • initiatingUserId "user1" (copied from event "e1" because the command is attached to "e1", FUTURE: copied from "e2")
  • initiatingTimestamp "t1'" (copied from event "e1" because the command is attached to "e1", FUTURE: copied from "e2")
  • causationId "causationId1" (copied from event "e1" because the command is attached to "e1", FUTURE: copied from "e2")
  • correlationId: "correlationId2"

• WorkspaceWasRebased

  • stream: "workspace-user1"
  • id "e8"
  • initiatingUserId "user1" (NOT copied!)
  • initiatingTimestamp "t4"
  • causationId (?)
  • correlationId: "correlationId2" (generated for this rebasing process)

Publishing of "user1" changes to "live"

• NodeAggregateWasCreated "home"

  • stream: "cs-live"
  • id "e9"
  • initiatingUserId "user1" (copied from event "e6")
  • initiatingTimestamp "t1" (copied from event "e6")
  • causationId "causationId1" (copied from event "e6")
  • correlationId: "correlationId3"

• NodeAggregateWasCreated "home/main" (tethered)

  • stream: "cs-live"
  • id "e10"
  • initiatingUserId "user1" (copied from event "e6" because the command is attached to "e1", FUTURE: copied from "e7")
  • initiatingTimestamp "t1'" (copied from event "e6" because the command is attached to "e1", FUTURE: copied from "e7")
  • causationId "causationId1" (copied from event "e6" because the command is attached to "e1", FUTURE: copied from "e7")
  • correlationId: "correlationId3"

• WorkspaceWasPublished

  • stream: "workspace-live"
  • id: "e11"
  • initiatingUserId "user1"
  • initiatingTimestamp "t5"
  • causationId (?)
  • correlationId: "correlationId3" (generated for this publishing process)

[bwaidelich]

After an issue @ahaeslich had today with lots of ContentStreamWasForked events without corresponding Workspace events (see Slack discussion) this missing feature gets more critical and made me think: The causationId does not have to be a UUID but it could contain some pre/suffix explaining the corresponding interaction, for example: rebaseWorkspace-62a2c5d06b834252b686e00ce6aa2548.

If it turns out to be too much work to introduce the complete causation/correlation handling for now, we should at least add some property to the ForkContentStream command (and the corresponding event), e.g.:

ForkContentStream::create(
    $rebasedContentStream,
    $baseWorkspace->currentContentStreamId,
    sprintf('Rebase workspace "%s" (content stream "%s"), $command->workspaceName->value, $command->contentStreamId->value),
)

[mhsdesign]

Ah this issue also made it to our list of big questions. But i dont really understand what is being discussed here, and it troubles me as this topic is marked critical ^^

[bwaidelich]

it troubles me as this topic is marked critical

Don't worry. This is critical for debugging the event log, but at least the last suggestion should be very easy to add and without any breaking changes

[bwaidelich]

If it turns out to be too much work to introduce the complete causation/correlation handling for now, we should at least add some property to the ForkContentStream command

As discussed, I created a separate high-prio issue (#4758) for that part and remove this one from the priority list

[bwaidelich]

Reminder: We should also set some kind of "originalEventId" metadata to events published to other content streams in order to correlate those.

This will allow us to detect "merges" rather than new events, i.e. instead of

%%{init: { 'logLevel': 'debug', 'theme': 'base', 'gitGraph': {'showBranches': true, 'showCommitLabel':true,'mainBranchName': 'Live', 'parallelCommits': false, 'rotateCommitLabel': true}} }%%
      gitGraph LR:
        branch "content stream A"
        commit id:"Node A created"
        commit id:"Node A properties set"
        commit id:"content stream A closed"
        branch "content stream B"
        commit id:"Content A stream forked'"
        commit id:"Node A created'"
        commit id:"Node A properties set'"
        commit id: "content stream B closed"
        branch "content stream C"
        commit id:"content stream B forked"
        commit id:"Node A created''"
        commit id:"Node A properties set''"
        checkout Live
        commit id:"Node A created'''"
        commit id:"Node A properties set'''"
        checkout "content stream C"
        commit id:"Content stream C removed"
        checkout "content stream B"
        commit id:"content stream B removed"

we could produce sth like

%%{init: { 'logLevel': 'debug', 'theme': 'base', 'gitGraph': {'showBranches': true, 'showCommitLabel':true,'mainBranchName': 'Live', 'parallelCommits': false, 'rotateCommitLabel': true}} }%%
      gitGraph LR:
        checkout Live
        commit id:"created"
        branch "content stream A"
        commit id:"Node A created"
        commit id:"Node A properties set"
        commit id:"content stream A closed"
        branch "content stream B"
        commit id:"content stream A forked'"
        merge "content stream A"
        checkout "content stream B"
        commit id: "content stream B closed"
        branch "content stream C"
        commit id:"content stream B forked"
        merge "content stream B"
        checkout Live
        merge "content stream C"
        checkout "content stream C"
        commit id:"content stream C removed"
        checkout "content stream B"
        commit id:"content stream B removed"