Implement dispatcher support in journal and object store

[alexguzun]

Depends on

• https://github.com/vlingo/vlingo-symbio/pull/14

[alexguzun]

@VaughnVernon Yes, I have seen that. That's why I persist the Dispatchable entries toghether with the Dispatchable: https://github.com/vlingo/vlingo-symbio-geode/pull/6/files#diff-32ad60fa2c99fe4b0f43a968cb2a2835R72

[DaveMuirhead]

Hi Alex -

Haven’t looked at your pull request yet, but is this task in the Release project still relevant given your pull request?

https://github.com/orgs/vlingo/projects/12#card-19781411

From a quick glance at your change set it looks maybe like you decided to persist Entries with Dispatchables in the same region (?).

Best regards,

Dave

---

Dave Muirhead Blue River Systems Group, LLC http://www.brsg.biz dave@brsg.biz 303-638-4618 Mobile 303-309-6242 Office 303-309-6243 Fax

On Jul 3, 2019, at 5:38 PM, Alex Guzun notifications@github.com wrote:

Depends on

vlingo/vlingo-symbio#14 https://github.com/vlingo/vlingo-symbio/pull/14 You can view, comment on, or merge this pull request online at:

https://github.com/vlingo/vlingo-symbio-geode/pull/6 https://github.com/vlingo/vlingo-symbio-geode/pull/6 Commit Summary

Implement dispatcher contract changes introduced in https://github.com/vlingo/vlingo-symbio/pull/14 Added the dispatcher support to GeodeObjectStoreActor File Changes

R src/main/java/io/vlingo/symbio/store/common/geode/GeodeQueries.java https://github.com/vlingo/vlingo-symbio-geode/pull/6/files#diff-0 (2) R src/main/java/io/vlingo/symbio/store/common/geode/dispatch/GeodeDispatchable.java https://github.com/vlingo/vlingo-symbio-geode/pull/6/files#diff-1 (23) A src/main/java/io/vlingo/symbio/store/common/geode/dispatch/GeodeDispatcherControlDelegate.java https://github.com/vlingo/vlingo-symbio-geode/pull/6/files#diff-2 (82) M src/main/java/io/vlingo/symbio/store/common/geode/pdx/GeodeDispatchableSerializer.java https://github.com/vlingo/vlingo-symbio-geode/pull/6/files#diff-3 (44) M src/main/java/io/vlingo/symbio/store/object/geode/GeodeObjectStoreActor.java https://github.com/vlingo/vlingo-symbio-geode/pull/6/files#diff-4 (137) D src/main/java/io/vlingo/symbio/store/state/geode/GeodeDispatcherControlActor.java https://github.com/vlingo/vlingo-symbio-geode/pull/6/files#diff-5 (126) M src/main/java/io/vlingo/symbio/store/state/geode/GeodeStateStoreActor.java https://github.com/vlingo/vlingo-symbio-geode/pull/6/files#diff-6 (53) R src/test/java/io/vlingo/symbio/store/common/MockObjectDispatcher.java https://github.com/vlingo/vlingo-symbio-geode/pull/6/files#diff-7 (55) A src/test/java/io/vlingo/symbio/store/common/event/Event.java https://github.com/vlingo/vlingo-symbio-geode/pull/6/files#diff-8 (20) A src/test/java/io/vlingo/symbio/store/common/event/TestEvent.java https://github.com/vlingo/vlingo-symbio-geode/pull/6/files#diff-9 (48) A src/test/java/io/vlingo/symbio/store/common/event/TestEventAdapter.java https://github.com/vlingo/vlingo-symbio-geode/pull/6/files#diff-10 (31) M src/test/java/io/vlingo/symbio/store/object/geode/GeodeObjectStoreTest.java https://github.com/vlingo/vlingo-symbio-geode/pull/6/files#diff-11 (132) M src/test/java/io/vlingo/symbio/store/state/MockObjectResultInterest.java https://github.com/vlingo/vlingo-symbio-geode/pull/6/files#diff-12 (14) M src/test/java/io/vlingo/symbio/store/state/geode/GeodeStateStoreActorTest.java https://github.com/vlingo/vlingo-symbio-geode/pull/6/files#diff-13 (24) Patch Links:

https://github.com/vlingo/vlingo-symbio-geode/pull/6.patch https://github.com/vlingo/vlingo-symbio-geode/pull/6.patch https://github.com/vlingo/vlingo-symbio-geode/pull/6.diff https://github.com/vlingo/vlingo-symbio-geode/pull/6.diff — You are receiving this because your review was requested. Reply to this email directly, view it on GitHub https://github.com/vlingo/vlingo-symbio-geode/pull/6?email_source=notifications&email_token=ABUJPIF5N7RFKXWGDDEYKPDP5UL4NA5CNFSM4H5LXZXKYY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4G5HRGZQ, or mute the thread https://github.com/notifications/unsubscribe-auth/ABUJPICYUR5A4DZVZTYVDTLP5UL4NANCNFSM4H5LXZXA.

[alexguzun]

Hi  @DaveMuirhead. Yes it's still relevant and you are right, I am persisting the entities ( and the state ) in the same region as Dispatchable. This allows reusing the same dispatchable persistence logic for all stores (Journal, Object and State)

[DaveMuirhead]

Ah, you said entities not entries. I understand. Thank you

---

Dave Muirhead Blue River Systems Group, LLC http://www.brsg.biz dave@brsg.biz 303-638-4618 Mobile 303-309-6242 Office 303-309-6243 Fax

On Jul 4, 2019, at 10:31 AM, Alex Guzun notifications@github.com wrote:

Hi @DaveMuirhead https://github.com/DaveMuirhead. Yes it's still relevant and you are right, I am persisting the entities ( and the state ) in the same region as Dispatchable. This allows reusing the same dispatchable persistence logic for all stores (Journal, Object and State)

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/vlingo/vlingo-symbio-geode/pull/6?email_source=notifications&email_token=ABUJPIARJHUNOEFWEM3VXYDP5YCVHA5CNFSM4H5LXZXKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZHSF6Y#issuecomment-508502779, or mute the thread https://github.com/notifications/unsubscribe-auth/ABUJPIGTRFKD6K2LDF6W6R3P5YCVHANCNFSM4H5LXZXA.

[alexguzun]

You are right. It's Entries.

      GeodeDispatchable<State<?>> instance = (GeodeDispatchable<State<?>>) o;
      out
      out
        .writeString("originatorId", instance.originatorId
        .writeObject("createdAt", instance.createdAt)
        .writeString("id", instance.id)
        .writeString("id", instance.id());
        .writeObject("state", instance.state);

      if (instance.state().isPresent()){
        out.writeObject("state", instance.state().get());
      }

      if (instance.entries()!=null && !instance.entries().isEmpty()){
        out.writeObjectArray("entries", instance.entries().toArray(new Entry[0]));
      }

[DaveMuirhead]

Do the Entries ever end up in their own table/region? Or do they remain persistent as part the Dispatchable?

---

Dave Muirhead Blue River Systems Group, LLC http://www.brsg.biz dave@brsg.biz 303-638-4618 Mobile 303-309-6242 Office 303-309-6243 Fax

On Jul 4, 2019, at 10:40 AM, Alex Guzun notifications@github.com wrote:

You are right. It's Entries.

  GeodeDispatchable<State<?>> instance = (GeodeDispatchable<State<?>>) o;
  out
  out
    .writeString("originatorId", instance.originatorId
    .writeObject("createdAt", instance.createdAt)
    .writeString("id", instance.id)
    .writeString("id", instance.id());
    .writeObject("state", instance.state);

  if (instance.state().isPresent()){
    out.writeObject("state", instance.state().get());
  }

  if (instance.entries()!=null && !instance.entries().isEmpty()){
    out.writeObjectArray("entries", instance.entries().toArray(new Entry[0]));
  }

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[alexguzun]

As part of Dispatchable.

[VaughnVernon]

@DaveMuirhead In trying to support reliable consumption of post-persistence state and events @alexguzun suggested harmonizing all of the symbio components, including the use of Dispatchable across all persistence types. There is a lot of dust in the air right now but it's the right approach and will pay dividends soon.

@alexguzun Is there any attempt at ordering of Dispatchable instances?

[alexguzun]

@VaughnVernon The Dispatchable instances are dispatched in the same order as write operations, that originate the instance. On redispatched the instances will be orderdered by io.vlingo.symbio.store.dispatch.Dispatchable#createdOn

[VaughnVernon]

@alexguzun Excellent. It may be that time order resolution is not perfect since many Dispatchable instances could be written in the same millisecond, but at least causal order (on most occasions) will be preserved. It's probably near to impossible that Dispatcables in the same long-running process would write from multiple entities in the same millisecond.

[DaveMuirhead]

@DaveMuirhead In trying to support reliable consumption of post-persistence state and events @alexguzun suggested harmonizing all of the symbio components, including the use of Dispatchable across all persistence types. There is a lot of dust in the air right now but it's the right approach and will pay dividends soon.

@alexguzun Is there any attempt at ordering of Dispatchable instances?

@VaughnVernon @alexguzun That all sounds good. The task regarding Geode transactions is relevant only if we need to persist entity state and associated entries in two separate Geode regions in a transactionally consistent way. If that's necessary (still) then there's details...Geode transactions require co-location of participating regions, etc.

[alexguzun]

@DaveMuirhead. Persisting Entries on the same region as dispatchable was a decision I have made just because the actual persistence in separate region was a task in progress. Other solution would be to persist just the ID's of Entries on dispatchable regions, and when retrieving the dispatch able, get the entries from their region.

[VaughnVernon]

@DaveMuirhead @alexguzun The Dispatchable persistence could be used to create a separate region with near-total-ordering of Entry instances. Basically it would be used the same way as Projection and Exchange feeds. It's eventually consistent, but I think from a DomainEvent (and persisted Command) standpoint this is more than acceptable.

[DaveMuirhead]

@DaveMuirhead @alexguzun The Dispatchable persistence could be used to create a separate region with near-total-ordering of Entry instances. Basically it would be used the same way as Projection and Exchange feeds. It's eventually consistent, but I think from a DomainEvent (and persisted Command) standpoint this is more than acceptable.

@VaughnVernon are you referring to the strategy we talked about a while back where, after persistence of the Dispatchable and its aggregated events/commands there is an asynchronous process (async event listener, most likely) that copies the domain events and commands from the Dispatchable into a separate region?

[VaughnVernon]

@DaveMuirhead Yes. Does that seem correct? Also a Long region singleton could be used to assign total ordering.

[DaveMuirhead]

@vaughnvernon that should work fine. Regarding the Long region, you’re referring to a single Long-valued sequence from which event id’s can be allocated, correct? If so, yes that was my thought and plan, as well.

[VaughnVernon]

@DaveMuirhead Maybe I have it wrong, but I thought that the Entry region would have a singleton Long. I didn't think that the Long would be in its own region (maybe you mistyped).

[DaveMuirhead]

@VaughnVernon I may not be understanding, but Geode doesn't have any built-in ID generator or auto-sequence key type. Keys and values are whatever type you choose. But there's a Long ID generator already checked in (io.vlingo.symbio.store.common.geode.identity.LongIDGenerator) and working, which is a variation of the one I've used for years. It relies on a region whose entries are sequence name / sequence value pairs. So, we'd have one entry in the ID sequences region for Entries that is the monotonically increasing, Long value that represents the last ID allocated. Hope that makes sense.

[VaughnVernon]

@DaveMuirhead Thanks, I think we are saying the same things. Graphically, I see it as two regions, one for Dispatchable and one for Entry:

Dispatchable: [c+ms=D] [r+ms=D] [f+ms=D] [a+ms=D] ... [s+ms=D]

Entry: [1=E] [2=E] [3=E] [w] [ID=4]

In Dispatchable the ?+ms means the unique id assigned to the D along with the time in milliseconds. The ? not being in lexicographic order indicates that the unique ids are not sorted, but the time in ms is used to provide an imprecise but "good enough" ordering, where "good enough" satisfies the causal ordering of D instances from the same Bounded Context and involved in the same distributed use case (long running process).

In Entry the [w] represents the next write/put key-value into the region. The ID is the key of the Long value that holds the next sequence number to be used to write where the w is.

I may be wrong but I have envisioned the writer into the Entry region to be a Geode Function so that each D value doesn't have to be read back to the application/service node in order to write to the Entry region.

[DaveMuirhead]

Oh, I see what you mean. Yeah, there’s no constraint that the keys and values of a region have to be homogenous, so could store the ID in the entry region. That’s fine.

On Jul 5, 2019, at 3:27 PM, Vaughn Vernon notifications@github.com wrote:

@DaveMuirhead https://github.com/DaveMuirhead Thanks, I think we are saying the same things. Graphically, I see it as two regions, one for Dispatchable and one for Entry:

Dispatchable: [c+ms=D] [r+ms=D] [f+ms=D] [a+ms=D] ... [s+ms=D]

Entry: [1=E] [2=E] [3=E] [w] [ID=4]

In Dispatchable the ?+ms means the unique id assigned to the D along with the time in milliseconds.

In Entry the [w] represents the next write/put key-value into the region. The ID is the key of the Long value that holds the next sequence number to be used to write where the w is.

I may be wrong but I have envisioned the writer into the Entry region to be a Geode Function so that each D value doesn't have to be read back to the application/service node in order to write to the Entry region.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/vlingo/vlingo-symbio-geode/pull/6?email_source=notifications&email_token=ABUJPIBGZDGTAFNFFDGF7R3P56ODXA5CNFSM4H5LXZXKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZKFZUI#issuecomment-508845265, or mute the thread https://github.com/notifications/unsubscribe-auth/ABUJPIFKAA4JOVMSJ6ZKAHLP56ODXANCNFSM4H5LXZXA.

[DaveMuirhead]

@alexguzun I’m not sure if you’re waiting for a review from me or if you have more to do given the build failures. Let me know. I have some work to do after you merge (not intended to rush or pressure you, just coordinating).

[alexguzun]

Hi @DaveMuirhead . The build is failing because it depends on https://github.com/vlingo/vlingo-symbio/pull/14.

Your review would be highly appreciated considering it has implications for you.

[VaughnVernon]

@DaveMuirhead I am leaving this open pending your review. I didn't find problems previously, but you may. The merge is yours to approve and perform.