Define a programming model/API for Event Sourcing

[sergeybykov]

It would help to have a general programming model for using Event Sourcing as an approach to managing grain state. Likely that would be an alternative or an extension to the declarative persistence model. The goal is to have a unified model that would allow different implementations (and against different data stores), so that an app can pick and choose which ones to use.

[sebastianburckhardt]

My current understanding is that Event Sourcing can be easily supported on top of the DataGrain API we have been discussing, which can in turn be done on top of various backplane services. Do you have links to particular samples that use the event sourcing pattern?

[richorama]

As a (simpleton) developer I'd like to implement an event sourced grain like this:

[EventStoreProvider("TableStorage")]
public MyEventGrain : EventGrain, IMyGrain
{
  int balance;
  public async Task Add(int value)
  {
    this.balance += value;
    await this.Commit();
  }
}

When Commit is called (which resides on the EventGrain base class) instead of persisting the current state (as a storage provider does) we record the method call (i.e. we record that add was called, and the values of the arguments). (or if you prefer to think of this in an actor context, you record the incoming message). If a method does not call commit, the event is never written (providing the ability to support 'readonly' methods).

When the grain is activated, the events recorded for that grain are then replayed by calling the methods again in sequence (although the Commit method would be disabled). This results in the grain's internal state being restored to where it was.

Note that the grain interface is not concerned with how this grain is implemented (whether it uses event sourcing or not).

I'm not sure what to do if the grain makes calls to other grains (this sounds like a bad idea).

I'm not sure if you would want to support a 'role up' or 'snapshot' of the state, to cater for grains with a large number of events.

Another consideration is whether you would want to allow a grain (or a system) to a restore to a previous point in history (i.e. play back all the events up to a certain point).

[sergeybykov]

@sebastianburckhardt We already have al least four different attempts to bring ES to Orleans: by @jkonecki, @yevhen, @ReubenBond and you. The purpose of this issue is try to come up with a programming model/API that would work for everyone while allowing for pluggable implementations.

I'm not sure what to do if the grain makes calls to other grains (this sounds like a bad idea).

@richorama I am offended you said calling other grains is a bad idea> :-) Seriously though, I read it as you are concerned about side effects. Rightfully so, of course.

This is where it's not clear to me if we should

1) view ES in the narrower scope of state updates, e.g. by providing a different persistence API where instead of the property bag and WriteStateAsync() would be a bunch of semantic update operations plus Commit(). This is roughly the direction @sebastianburckhardt and @jkonecki took. With this approach we are not concerned about side effects because we don't replay grain calls, and only reconstitute the persistent state object.

or

2) use it as a 'grain calls log', which is my understanding of @ReubenBond's and @yevhen's approach and your example above.

It is also very possible that I'm too confused by all these similar but different stories.

[sebastianburckhardt]

I think we can agree that at some point, the runtime has to be able to serialize/deserialize (1) the grain state, and (2) an update descriptor. Requirement (1) was always there for persistent grain, but requirement (2) is new.

The question is if this should be automatic (done by compiler or at runtime using reflection), or under explicit programmer control. Today, the persistence mechanism is not automatic. The programmer explicitly provides a serializable class that defines the grain state. There are advantages and disadvantages to that.

If we are doing the same thing to serialize updates, i.e. go with explicit programmer control, the user would write serializable classes to define all the update operations. I show an example below. There is a bit more redundancy in this code than I like, unfortunately. The cool part is that since updates are now first-class, we can support very useful and powerful APIs involving updates and update sequences (e.g. we can query which updates are currently still pending and not yet confirmed, we can get an IEnumerable of all updates, and we can subscribe to a stream of updates).

[Serializable]
class MyGrainState
{
   public int Balance {get; set;}
}

[Serializable]
class MyAddOperation : IUpdates<MyGrainState> {
   public int Amount { get; set; }
   public void Update(MyGrainState state) {
      state.Balance += Amount;
   }
}

class MyGrain : DataGrain<MyGrainState>
{ 
    public async Task Add(int value)
    {
         await QueueUpdate(new MyAddOperation() { Amount = value });
    }

}

[ReubenBond]

Some people were confused by my Event Sourcing implementation. I had a chat with @gregoryyoung, which hopefully cleared up most of that confusion and also helped to refine the implementation in my mind.

Framework support for Event Sourcing is a great idea! I am very confident that we can come up with an API which is both sound, easy to use, and (hopefully) which makes it obvious when it's being abused.

Here is one idea: we have command methods which do not directly emit events, event methods which do directly emit events, and query methods which are read-only.

Command methods take their arguments, perform required computation on them, collect whatever ambient information is required, and then call Event methods. Command methods are not replayable. Here is an example of a very basic Command method:

public Task PlaceOrder(Order order, User user)
{
  // Gets the taxation rate at the time the order is placed.
  var tax = await taxActor.GetApplicableTax(order, user.Address);
  return this.OrderPlaced(order, user, tax);
}

Event methods perform validation of their arguments, emit the event, and then apply the event. Event methods are replayable. Validation is only performed once: before the event is persisted. During replay, the event is not re-validated. Here is an example of an Event method:

public Task AddedUser(User user)
{
  return this.Commit(
    validate: () =>
    {
      if (user == null)
      {
        throw new ArgumentNullException("user");
      }
    }, 
    apply: () => this.users.Add(user.Id));
}

Of course, if no validation is required, that method becomes very terse:

public Task AddedUser(User user)
{
  return this.Commit(() => this.users.Add(user.Id));
}

Query methods are exactly as they are today. Here is a Query method:

public Task<int> GetUserCount()
{
  return this.users.Count();
}

Alright, so those are the kinds of methods which we can have, but what if you need to emit multiple events atomically? @gregoryyoung gives an example of a stock trading platform where:

1. An order is placed for 1000 shares
2. The order is partially filled (200 shares)
3. Another order is placed for the remaining 800 shares

Events 2 & 3 should be committed atomically. I propose we do this within the scope of a using block within a command method. This could even be nested (but let's not get into that). Example of committing multiple events (pseudo code):

using (var tx = this.CreateEventTransaction())
{
  await this.TradeOccurred(buyer, seller, symbol, volume, price);
  await this.PlaceOrder(buyer, symbol, remainingVolume, price);
  await tx.Commit();
}

This is surprisingly easy to implement: each call to CreateEventTransaction pushes an in-memory IEventJournal to the grain's stack of event journals. tx.Commit() commits the events in tx's journal to the previous journal. The in-memory journal's Dispose() method pops it from the stack.

Let me know what you think. We will discuss versioning of event handlers as well as event method arguments. We can also discuss views/projections, which can very nicely plug into Orleans' streaming system :smile:!

Oh, also, we can consider creating Roslyn Code Analyzers to guide users onto the right track (eg, event methods should have exactly one statement: this.Commit(...)). This can also help us to warn users when they're performing side effects in their apply call.

Commands and events are identical if the command performs no computation & require no validation (I think this is what made @yevhen decide I had implemented command sourcing).

Message sourcing can be implemented also: you Commit() query methods. The apply parameter in Commit(...) can return a value, but that's not necessary.

Very excited!

[gabikliot]

I think I like it a lot. Basically, the way I understood what you wrote, one can have the full flexibility: you decide what should be sourced (journalled) and what not. You may have all methods journalled, or only some. You can emit multiple events in a method, as tx or not, or none. The only restriction is that what ever you journal does not have side effects - it only performs transformations on the in memory state (or zero transformations and returns a value). Plus optional validation. If I understood it correctly, then I like it! I wonder however what @yevhen has to say. :-)

Did not understand the connection between views/projections and Orleans' streaming, but that is orthogonal.

I am also super interested in: "consider creating Roslyn Code Analyzers to guide users onto the right track (eg, event methods should have exactly one statement: this.Commit(...) ). " If it is possible to create a compile time validation of certain complex, context aware patterns, this would be super useful in a lot of places, unrelated to ES! One example is check that grain code never calls Task.Wait or Task.Result. We have more examples like this

[ReubenBond]

Your understanding is correct.

I'm trying to keep from discussing everything at once, but basically the event journal could be subscribed to. Subscribers would implement the same interface as the grain (or at least he journalled part of it). That way, the subscriber can handle each of the events in a natural, .NET-native manner. Does that make sense? Does it sound like a good approach?

Regarding Roslyn Code Analyzers, I'm referring to this: https://channel9.msdn.com/Events/dotnetConf/2015/NET-Compiler-Platform-Roslyn-Analyzers-and-the-Rise-of-Code-Aware-Libraries I'm not certain how feasible Code Analyzers are, yet - it might be a pipe-dream.

Edit: more detail on analyzers: https://msdn.microsoft.com/en-us/magazine/dn879356.aspx

[gregoryyoung]

Read the views thread on akka.persistence.

On Sat, Apr 18, 2015 at 8:00 AM, Reuben Bond notifications@github.com wrote:

Your understanding is correct.

I'm trying to keep from discussing everything at once, but basically the event journal could be subscribed to. Subscribers would implement the same interface as the grain (or at least he journalled part of it). That way, the subscriber can handle each of the events in a natural, .NET-native manner. Does that make sense? Does it sound like a good approach?

Regarding Roslyn Code Analyzers, I'm referring to this: https://channel9.msdn.com/Events/dotnetConf/2015/NET-Compiler-Platform-Roslyn-Analyzers-and-the-Rise-of-Code-Aware-Libraries I'm not certain how feasible Code Analyzers are, yet - it might be a pipe-dream.

— Reply to this email directly or view it on GitHub https://github.com/dotnet/orleans/issues/343#issuecomment-94127882.

Studying for the Turing test

[ReubenBond]

@gregoryyoung do you mean this thread? https://groups.google.com/forum/#!searchin/akka-dev/views/akka-dev/dhEQWEeqY40/AOW1PKgFRHAJ

[gabikliot]

Got it about looking on the journal as stream of events. Nice! Thanks @gregoryyoung, will do. That:http://doc.akka.io/docs/akka/snapshot/scala/persistence.html?

[ReubenBond]

If that's the thread, thank you - we could integrate the notion of stream hooks into our model. We would provide a handler which is executed on each journalled message.

That gives users a chance to emit the event to a stream of their choosing - that allows for heterogeneous partitioning, rather than forcing the partition to be per-actor. So I could fork the stream for all user actors, and have security-related events emitted to a special audit log. Or if we are message sourcing, we could copy all queries to a stream for analytics.

[gregoryyoung]

Akka Persistence on the Query Side

On Sat, Apr 18, 2015 at 8:04 AM, Gabriel Kliot notifications@github.com wrote:

Got it about looking on the journal as stream of events. Nice! Thanks @gregoryyoung https://github.com/gregoryyoung, will do. That: http://doc.akka.io/docs/akka/snapshot/scala/persistence.html?

— Reply to this email directly or view it on GitHub https://github.com/dotnet/orleans/issues/343#issuecomment-94127969.

Studying for the Turing test

[ReubenBond]

Alright, got it: https://groups.google.com/forum/#!topic/akka-user/MNDc9cVG1To

I'm wary of making users pay for something they might not be using, and generalized query support can be very expensive. A very simple hooking point solves the problem efficiently, but we should carefully consider how we support querying and stream discovery. I think the question of query/discovery is broader than just ES, though: people often ask how to search for grains by some property.

[witterlee]

Very happy to see a topic of Event Sourcing I like this point. the event can be publish to event-stream and any interested actor can subscribe this event.

...but basically the event journal could be subscribed to..

[yevhen]

I'm trying to keep from discussing everything at once, but basically the event journal could be subscribed to. Subscribers would implement the same interface as the grain (or at least he journalled part of it). That way, the subscriber can handle each of the events in a natural, .NET-native manner. Does that make sense? Does it sound like a good approach?

No, it doesn't. That will lead to code bloat in cases where subscribers are only interested in selection of events, not the whole set.

Also, your idea doesn't mesh well with newly introduced Streams api, which is based on message-passing. Basically, with your idea - we're back to Observer api.

[jkonecki]

I would like to start by stating that I would prefer to implement event sourcing rather than command sourcing, as described by @richorama. Command sourcing can be tricky when it comes to side-effects, like communication with external services, which shouldn't happen during replay. Also, command sourcing fails to capture the fact whether the command succeeded or not, which means it's difficult to subscribe to stream externally.

Event sourcing is defined many times by @gregoryyoung as derivation of the current state from the stream of past events. As I've mentioned during the first meetup, the separation of the grain state and grain itself is very handy here. In my opinion t is the state that is event sourced, not the gain. I see the grain more as a repository of logic. I fully agree with @richorama that the usage of the events-sourced state should be as similar to the current way gain interacts with its state as possible,

Here is @richorama example rewrote the way I see it:

[EventStoreProvider("TableStorage")]
public MyEventGrain : EventGrain, IMyGrain
{
  public async Task Add(int value)
  {
    // instead of this.State.Balance += value;
    this.State.RaiseEvent(new BalanceIncreased(value));

   // no change to current api
    await this.State.WriteStateAsync();
  }
}

I believe that the events should be explicitly defined and not auto-generated from grain's method arguments. Events are the fundamental part of domain modelling and may contain more information than is passed to the method that raises them. For example an ConfirmationEmailSent event may contain an email unique identifier returned from EmailSending service invoked from grain method and not passed as an argument to the method.

One benefit of event sourcing is the ability to rehydrate the aggregate as of a certain version, for example for debugging purposes. In order to achieve this in Orleans we would need a way to obtain a reference to the grain by passing a certain version. Right now I don't want to go into the detail of if the version is a number, string or etag. I'm just pointing out that there should be a way to obtain multiple instances of the same grain with different states. I'm guessing this with probably result with the grain reference being extended in order to differentiate between multiple instances of the same grain.

Another feature which I believe is required is the ability to publish raised events. A plugin architecture here would be perfect with providers for EventHub, Table Queues and ServiceBus being the most obvious.

[yevhen]

+1 to @jkonecki

I'm not going to support the idea of having implicit generic events auto-generated from method signatures. It's too magical, non-conformist and non-idiomatic. It creates completely unnecessary (accidental) complexity.

There other ways exist to deal with verbosity of declaring events, such as using custom DSL or simply using the DSL provided by serialization library (.proto, .bond).

[yevhen]

One benefit of event sourcing is the ability to rehydrate the aggregate as of a certain version, for example for debugging purposes. In order to achieve this in Orleans we would need a way to obtain a reference to the grain by passing a certain version. Right now I don't want to go into the detail of if the version is a number, string or etag. I'm just pointing out that there should be a way to obtain multiple instances of the same grain with different states. I'm guessing this with probably result with the grain reference being extended in order to differentiate between multiple instances of the same grain.

I think for that you don't need to go through the framework (runtime). Just create an instance of your (presumably) POCO aggregate-actor and replay its stream on it. Use-case that you have presented should not require runtime.

[yevhen]

@sergeybykov

2) use it as a 'grain calls log', which is my understanding of @ReubenBond's and @yevhen's approach and your example above.

Absolutely opposite. I view event sourcing as persistence mechanism. So I'm more on side of @jkonecki and rest of 100500 folks from ES community :)

[yevhen]

@sergeybykov

See example here https://github.com/yevhen/Orleankka/blob/master/Source/Example.EventSourcing/Domain.cs. I don't record (log) calls to the grain, I record state changes (transitions) as events, which do capture a business intent of a change.

See https://github.com/yevhen/Orleankka/blob/master/Source/Example.EventSourcing/Domain.cs#L46 as an example of something very casual - capturing more data than just an input.

[yevhen]

Guys, from most of the comments here, I can deduce that many of you do not understand what Event Sourcing actually is.

I see no point discussing it any further until we're on the same page. I don't like playing Chinese whispers.

Please, take your time and watch at least first quarter of https://www.youtube.com/watch?v=whCk1Q87_ZI then I believe we can have a more productive discussion.

[ReubenBond]

@yevhen I get the feeling you haven't read the thread or the JabbR conversation. You told Greg Young to come and tell me that this is a bad idea and that this is not event sourcing and it's wrong. He and I had a chat, he indicated that yes, this library supports event sourcing. We can also do command and message sourcing.

We are not blindly recording calls to grains. We validate the arguments, we have the option to collect environmental information at the time of the event, and to perform computations on whatever data before persisting and applying the event. That is all clearly demonstrated above.

This proposal is much more in-tune with the rest of Orleans, in my opinion. Your system is easily implemented using this proposal. You would have a single event handler which takes arguments deriving from some kind base type with an Apply() method. You can consume this library and drop the static types as you prefer - just like how you consume Orleans and drop static types, making it more like Akka. It's fine to take your approach. We can support both elegantly.

It's much easier to throw away statically checked types than to keep them. Once you throw them away, they're gone.

I've watched Greg's videos, but thanks for the link. This one is good, too.

[veikkoeeva]

I'm uncomfortable using event sourcing terms and I'm somewhat loss with it, but I'll note that for what I know, this looks either a form of CEP or a way to persistent events (in general fashion, i.e. without business specific structure?). Taking a cue from the linked Google thread and a question by Prakhyat Mallikarjun, whatever we do, I'd like to have an easy route to business specific queries and storage.

I think I see the value as a pattern on how to deal with events as such, especially when CEP like functionality isn't needed.

The problems I'm usually dealing with in the streaming domain are CEP like queries, how to make them fault-tolerant, how to quickly get aggregates and initialize state and this when there might be multiple streams to merge. The rest, like sequence identifiers, have always been business specific. For instance sensor readings are timestamped or have a sequence ID defined by the source. Then there is another timestamp applied when the event is received on the server and at this point their relative order, from a business perspective, doesn't matter as only timestamps are looked (and perhaps corrected for known anomalies). Considering events flowing from a metering field, they come through multiple socket ports through multiple servers (applying a consistent sequence number doesn't look like sensible), get routed to the a same event stream and persisted. Ack for receiving the event is given when it's on a durable store (just after receiving it or when persisted to a db).

What I'd like to do, many times, is to calculate aggregates like sums or cumulative sums on the fly and persist these along the data storage and use these to initialize the streams quickly upon recovering from interruptions. There might be some heavier analytics and/or data enrichment done at the data storage, which would be nice to join back to the event stream.

What I do currently for my little demo for myself is that upon ingesting data I immediately save it on the edge and then pass along for further domain-specific processing and then save that data to domain specific store and then put to transport streams for further consumption. If further processing crashes before some other save point, I'll replay the data either from the initial save location or the domain specific location. This is a kind of a stream, source of which can be seen in either of the two save locations. I can imagine this getting difficult with more downstream consumers and aggregation logic. For instance, if further downstream aggregates are being calculated, I might like to save it to the domain specific store along with the events that defines the aggregate.

For a more "application logic", there's a good example on what I've thought about at work for some years. Getting #er blog, functional event-sourcing – compose.

In any event, how I see what I'd like to have. Carry on. :)

<edit: I might add that currently I save all data explicitly without using the Grain state mechanism.

[gregoryyoung]

" The problems I'm usually dealing with in the streaming domain are CEP like queries, how to make them fault-tolerant, how to quickly get aggregates and initialize state and this when there might be multiple streams to merge. The rest, like sequence identifiers, have always been business specific. For instance sensor readings are timestamped or have a sequence ID defined by the source. Then there is another timestamp applied when the event is received on the server and at this point their relative order, from a business perspective, doesn't matter as only timestamps are looked (and perhaps corrected for known anomalies). Considering events flowing from a metering field, they come through multiple socket ports through multiple servers (applying a consistent sequence number doesn't look like sensible), get routed to the a same event stream and persisted. Ack for receiving the event is given when it's on a durable store (just after receiving it or when persisted to a db). "

Perhaps metering devices are not the only use case.

I have found many threads just like this one. People tend to think of one use case they care about and not of the many thousands that exist. You seem to suggest linearization is a dumb idea that never works. There are thousands of production systems that disagree with you. At times ordering is important. There are many ways of achieving it and they have trade offs.

On Sat 18 Apr 2015 at 21:18 Veikko Eeva notifications@github.com wrote:

I'm uncomfortable using event sourcing terms and I'm somewhat loss with it, but I'll note that for what I know, this looks either a form of CEP or a way to persistent events (in general fashion, i.e. without business specific structure?). Taking a cue from the linked Google thread https://groups.google.com/forum/#!topic/akka-user/MNDc9cVG1To and a question https://groups.google.com/d/msg/akka-user/MNDc9cVG1To/IO9DoBTMJ3kJ by Prakhyat Mallikarjun, whatever we do, I'd like to have an easy route to business specific queries and storage.

I think I see the value as a pattern on how to deal with events as such, especially when CEP like functionality isn't needed.

The problems I'm usually dealing with in the streaming domain are CEP like queries, how to make them fault-tolerant, how to quickly get aggregates and initialize state and this when there might be multiple streams to merge. The rest, like sequence identifiers, have always been business specific. For instance sensor readings are timestamped or have a sequence ID defined by the source. Then there is another timestamp applied when the event is received on the server and at this point their relative order, from a business perspective, doesn't matter as only timestamps are looked (and perhaps corrected for known anomalies). Considering events flowing from a metering field, they come through multiple socket ports through multiple servers (applying a consistent sequence number doesn't look like sensible), get routed to the a same event stream and persisted. Ack for receiving the event is given when it's on a durable store (just after receiving it or when persisted to a db).

What I'd like to do, many times, is to calculate aggregates like sums or cumulative sums on the fly and persist these along the data storage and use these to initialize the streams quickly upon recovering from interruptions. There might be some heavier analytics and/or data enrichment done at the data storage, which would be nice to join back to the event stream.

What I do currently for my little demo for myself is that upon ingesting data I immediately save it on the edge and then pass along for further domain-specific processing and then save that data to domain specific store and then put to transport streams for further consumption. If further processing crashes before some other save point, I'll replay the data either from the initial save location or the domain specific location. This is a kind of a stream, source of which can be seen in either of the two save locations. I can imagine this getting difficult with more downstream consumers and aggregation logic. For instance, if further downstream aggregates are being calculated, I might like to save it to the domain specific store along with the events that defines the aggregate.

For a more "application logic", there's a good example on what I've thought about at work for some years. Getting #er blog, functional event-sourcing – compose http://gettingsharper.de/2015/02/13/functional-event-sourcing-compose/.

In any event, how I see what I'd like to have. Carry on. :)

— Reply to this email directly or view it on GitHub https://github.com/dotnet/orleans/issues/343#issuecomment-94188794.

[ReubenBond]

We want an API which supports {Message,Command,Event} Sourcing. We can call it our Journaling API.

Here are a few desirable properties of the API:

1. Easy to consume: Orleans is a framework for simplifying the development of large scale, distributed systems. The API should be in-tune with this. This means enabling journaling via constructs & techniques which are familiar to what your typical .NET developer is comfortable with. If Event Sourcing is a hassle, then guess what? Developers will avoid it. We don't want that. It is our job to do the heavy lifting to make their lives easier. Every line of code we save them is a point for us. Every time we save them from a programming error, that's another point for us.
2. Extensible: No one's arguing here. Let's take this as an opportunity to begin support for dependency injection.
3. Support for projections: By supporting multiple views of the event log, we allow consumers to more easily build search, analytics, auditing, billing, & many other features. We should have the ability to consume the raw log as well as support statically typed event subscription.

Supporting statically typed events is easiest if an implementation of the event sourced interface is passed to the subscribe method. That works cleanly, and you can argue for it, but @yevhen rightly points out that many of the interface methods would be often go unimplemented. One option to remedy this is to let consumers decompose their interface so that they only need to implement a subset of it - allowing the processing of events from different actor types. Alternatively, they can filter the raw event stream. Alternatively we can see what possibilities Roslyn gives us for checking type assertions at development & compile time.

@yevhen's approach more cleanly supports projections, but is verbose in the normal case, where users are logging & applying events. So it's a trade-off. This is basically the object / lexical closure equivalency debate. Orleans as a framework clearly leans towards objects, whereas Akka leans towards closures. Both options have merits, both have downsides, but let's do our best to be consistent.

[gregoryyoung]

It being we are discussing method = message = schema, its true they are all ways of representing schema though they have varying sets of tradeoffs ... You have found one of these trade offs so far in dealing with projection code that needs to read events. Another that would be worth thinking about is heterogenous models. There is not a right answer without context.

Why not just make this a choice? The same underlying "framework" code would work regardless of how you choose to represent schema having code capable of reading schema via type or via a method definition should be relatively trivial to support.

On a side note on "making types being a hassle" as a form of defining schema is a common argument I have heard with this style (messaging in general) often by people who are new to such systems. Over time the argument tends to go away. Whether I define thing through method based schema, type base schema, or through an actual schema like .proto files makes such a tiny difference.

On Sun, Apr 19, 2015 at 3:59 PM, Reuben Bond notifications@github.com wrote:

We want an API which supports {Message,Command,Event} Sourcing. We can call it our Journaling API.

Here are a few desirable properties of the API:

1.

Easy to consume: Orleans is a framework for simplifying the development of large scale, distributed systems. The API should be in-tune with this. This means enabling journaling via constructs & techniques which are familiar to what your typical .NET developer is comfortable with. If Event Sourcing is a hassle, then guess what? Developers will avoid it. We don't want that. It is our job to do the heavy lifting to make their lives easier. Every line of code we save them is a point for us. Every time we save them from a programming error, that's another point for us. 2.

Extensible: No one's arguing here. Let's take this as an opportunity to begin support for dependency injection. 3.

Support for projections: By supporting multiple views of the event log, we allow consumers to more easily build search, analytics, auditing, billing, & many other features. We should have the ability to consume the raw log as well as support statically typed event subscription.

Supporting statically typed events is easiest if an implementation of the event sourced interface is passed to the subscribe method. That works cleanly, and you can argue for it, but @yevhen https://github.com/yevhen rightly points out that many of the interface methods would be often go unimplemented. One option to remedy this is to let consumers decompose their interface so that they only need to implement a subset of it - allowing the processing of events from different actor types. Alternatively, they can filter the raw event stream. Alternatively we can see what possibilities Roslyn gives us for checking type assertions at development & compile time.

@yevhen https://github.com/yevhen's approach more cleanly supports projections, but is verbose in the normal case, where users are logging & applying events. So it's a trade-off. This is basically the object / lexical closure equivalency http://c2.com/cgi/wiki?ClosuresAndObjectsAreEquivalent debate. Orleans as a framework clearly leans towards objects, whereas Akka leans towards closures. Both options have merits, both have downsides, but let's do our best to be consistent.

— Reply to this email directly or view it on GitHub https://github.com/dotnet/orleans/issues/343#issuecomment-94274774.

Studying for the Turing test

[yevhen]

what your typical .NET developer is comfortable with

Typical Joe, doesn't build distributed systems which use event sourcing. He builds CRUD apps using his favorite RDB/ORM combo.

@richorama

As a (simpleton) developer I'd like to first gain an understanding of what event sourcing is, whether I really need it, what are the trade-offs and issues associated with it, before I even try to convince my manager.

As a (simpleton) developer, I first need to understand what distributed systems are, whether do I need to build the one and take all risks associated with it. That means, I first need to understand CAP, storage options and associated trade-offs, read some or all papers listed here and learn about distributed scalable architectures alternative to my simpleton RDB\ORM combo, such as REST, EDA, CQRS, ETC.

As a (simpleton) developer, before diving into a brave new world of distributed systems programming, it definitely won't hurt, if I first learn crucial design & modeling principles and patterns, like DDD, NoSQL, EIP, ETC. By doing that, I will avoid constantly begging authors of super-powerful distributed actor frameworks to support cross-actor (cross-partition) transactions, because I won't be modeling my actors after each row in my RDB, since at that moment I'll already have a firm understanding of what Aggregate is.

As a (simpleton) developer, by doing my homework and not being stupid, I'll get a chance to avoid shooting myself in a foot, by thinking that all of my (scalability, performance, etc) problems could be somehow auto-magically solved by technology/framework/unicorn alone, without requiring me making any investments in learning about the hard STUFF at all.

As a (simpleton) developer, at that moment, without any doubt, I'll be understanding that:

• Events happen in the past. For example, "the speaker was booked," "the seat was reserved," "the cash was dispensed." Notice how we describe these events using the past tense.
• Events are immutable. Because events happen in the past, they cannot be changed or undone. However, subsequent events may alter or negate the effects of earlier events. For example, "the reservation was cancelled" is an event that changes the result of an earlier reservation event.
• Events are one-way messages. Events have a single source (publisher) that publishes the event. One or more recipients (subscribers) may receive events.
• Typically, events include parameters that provide additional information about the event. For example, "Seat E23 was booked by Alice."
• In the context of event sourcing, events should describe business intent. For example, "Seat E23 was booked by Alice" describes in business terms what has happened and is more descriptive than, "In the bookings table, the row with key E23 had the name field updated with the value Alice."
• Event sourcing is a way of persisting your application's state by storing the history that determines the current state of your application. For example, a conference management system needs to track the number of completed bookings for a conference so it can check whether there are still seats available when someone tries to make a new booking.

Source: CQRS Journey. Introducing Event Sourcing

Dear Joe (typical senior assistant of junior .NET developer), I care because you do ...

[jkonecki]

+1 for @yevhen definition of event sourcing above.

[ReubenBond]

I get that you don't care about newbies, @yevhen, but go an read the Orleans MSR site, papers, and watch some talks on it. Non-experts are the target audience. A major goal of Orleans is to simplify the development of distributed systems.

[veikkoeeva]

@gregoryyoung

Perhaps metering devices are not the only use case.

I have found many threads just like this one. People tend to think of one use case they care about and not of the many thousands that exist. You seem to suggest linearization is a dumb idea that never works. There are thousands of production systems that disagree with you. At times ordering is important. There are many ways of achieving it and they have trade offs.

I apologize if I came across as ignorat. I assure I'm in good good faith here and try to fit this to my mental frame.

I saw the videos of @ReubenBond and @yevhen and a simple API such as that is desirable. Alas, I haven't read much about event sourcing, but it seems like coming to close things I have done. The terms and concepts aren't totally alien, but it's safest to say it hasn't been in the center of my radar, so to speak. Maybe one way to see this is in some way like finding items in a collection, there are .First, Contains etc. in the framework and collection classes, and LINQ. One then needs to pick the right tool (and learn what is the right tool). In this case there is the API and a simple set of semantics behind it (like immutable events that have happened in the past).

Maybe the following helps to see an edge when it comes people like me and gives better tools to explain. What comes to linearization and things I've seen are more to do with integration suites – or queuing systems. I see integration suites, and queuing, solve a set of overlapping problems, and integration suites especially some "political enterprise problems" (they span multiple departments) as well as providing various orchestration logics. Then there are various gateways (like SMS ones), which have these concepts of linearization too, but come with an attached bag one wants to have reports from the gateway to match with what the other end sees (e.g. to match the records to see billing is correct or to troubleshoot if customer tells she hasn't received a message).

This probably is frustrating. Were you to explain this to an embedded developer, she would understand integration likely very differently. But I'll watch this thread and educate myself.

[veikkoeeva]

@ReubenBond Spot on. Simple, robust and focus on throughput (and some latency). That's what interests me (*). I would say it's a worthwhile goal to have as much compiler help as possible. A good selling point for event sourcing is an audit trail functionality you had in your presentation.

We (at work, not using Orleans, Orleans is something I do on my own time) are currently doing a project where we are auditing on table level (checkpointing contents before and after operation to an audit trail) and the user ID comes from IdSrv. We use IdSrv with both external usernames and internal ADFS (with possibly federation). Our concern for the forseeable future is to see who has changed or touched upon certain data (locked down schemas all the way from AD), but storage level auditing doesn't translate into actual trail that easily – considering this particular scenario. Showing a scenario like you showed that's mostly "out of the box" is a worthy one, I think. There are some things then, such that the audit trail would be secured somewhere in some possibly predefined format, but I suppose that's not a problem. The bigger problem is having a robust API and a pattern which is easy to follow and apply within reasonable limitations (not fighting the framework, falling into the pit of success etc.).

(*) I understand some of the theory (I'm just about to start this and yes, I've read a ton about architectures and other concepts, but this particular topic has for a reason or another evaded me) and I have built some kind of distributed systems.

[sergeybykov]

@evhen Sorry, I mischaracterized your approach. I guess I conflated your unified interface with command sourcing.

@jkonecki's and @yevhen's descriptions match my view of the basic ES as a way of managing a grain's state as a sequence of updates. I like that it is isolated and is not subject to side effects. I like events explicitly defined as they, like @yevhen stated, codify business intent. Hence I don't think they should be equated with actor messages (grain calls). I think we can provide a simple API that will be easy to understand for a broad population of developers (@reubenbond correctly emphasized that this continues to be a priority for us), and @jkonecki showed as example of what we could do. We could still do some codegen if it helps with boilerplate, but in my opinion it should be a derivative of explicitly defined types/operations, sort of like we do for declarative persistence today.

What's not clear to me if we want to make event deduplication a first class feature of the API by requiring an explicit unique event ID (transaction ID) instead of relying on application passing it a as a property if an app-specific data object. But I think this is a second level design question.

Can we look at this level of functionality as the foundation layer for more advanced features like command sourcing (grain call log), queries, and subscription to updates? What I mean by that is that with the barebones ES we would already get a mechanism for auditing, concurrency, deduplication, and compensations. If command sourcing can be layered on top, that can be extra goodness, but it wouldn't be required in order to get the mentioned ES benefits.

The query part is much less clear to me. I need to read and think more. But it seems to me, maybe me being naïve, that it also can be easily layered on top of the barebones ES.

[jkonecki]

@sergeybykov Sergey, I assume you would like to see an IEventStore interface with a selection of implementations, Table Storage being one of them. I have a Table Storage example based on CQRS Journey (https://msdn.microsoft.com/en-us/library/jj554200.aspx) with updated Storage SDK and async/await. I still need to polish it a bit.

[sergeybykov]

@jkonecki Yes, definitely.

Just as an example, a major customer of Orleans uses a hand-coded library that saves state updates as a sequence of append-only records in an Azure Table partition, where partition key is the actor ID and row key is a unique update ID (transaction ID). That way they deduplicate updates at the storage level. In addition, they transactionally write a snapshot row, so that that an event append and a snapshot update are executed atomically (under the same partition key).

[jkonecki]

@sergeybykov this is exactly what I'm doing. Row Id is an incremented version number which gives ordering and optimistic concurrency.

[sergeybykov]

@jkonecki You are not doing the snapshot part, are you?

[jkonecki]

No, I haven't implemented snapshots but adding a memento pattern will be very simple. Honestly with Grains being cached in memory the need of snapshots is even smaller than in systems where aggregates are dehydrated on every command.

[gregoryyoung]

In memory is a snapshot

On Mon, Apr 20, 2015 at 6:55 PM, Jakub Konecki notifications@github.com wrote:

No, I haven't implemented snapshots but adding a memento pattern will be very simple. Honestly with Grains being cached in memory the need of snapshots is even smaller than in systems where aggregates are dehydrated on every command.

— Reply to this email directly or view it on GitHub https://github.com/dotnet/orleans/issues/343#issuecomment-94492190.

Studying for the Turing test

[jkonecki]

@gregoryyoung Yes ;-). I think @sergeybykov was asking for persisted snapshots.

[gregoryyoung]

No need to build them in the mdoel they can be trivially added on top

On Mon, Apr 20, 2015 at 7:04 PM, Jakub Konecki notifications@github.com wrote:

@gregoryyoung https://github.com/gregoryyoung Yes ;-). I think @sergeybykov https://github.com/sergeybykov was asking for persisted snapshots.

— Reply to this email directly or view it on GitHub https://github.com/dotnet/orleans/issues/343#issuecomment-94494226.

Studying for the Turing test

[jkonecki]

@gregoryyoung what do you mean by 'in the model'? I expect snapshots can be handled by the Orleans framework / storage provider so this want leak to the application code.

[gregoryyoung]

eg there is no reason to do snapshots in the event sourced model/api they can be bolted on trivially on top

On Mon, Apr 20, 2015 at 7:12 PM, Jakub Konecki notifications@github.com wrote:

@gregoryyoung https://github.com/gregoryyoung what do you mean by 'in the model'? I expect snapshots can be handled by the Orleans framework / storage provider so this want leak to the application code.

— Reply to this email directly or view it on GitHub https://github.com/dotnet/orleans/issues/343#issuecomment-94496529.

Studying for the Turing test

[jkonecki]

Oh, I see. Yes.

[sergeybykov]

Like I said, it was just an example of what our customers do today, and not necessarily what we have to implement. :-)

I presume we'll need snapshots for performance under the covers in some form anyway, won't we?

[yevhen]

Snapshots, is an optimization technique. Will give you good ROI only for actors with really long lifecycle and heaps of generated events. I found it might be cheaper to just replay events. Snapshots require versioning and it's generally a hassle.

Long lived, hot actors will be residing in-memory most of the time (even, maybe pinned), so constant time/resource waste on load/restore/replay should not be a concern here.

[sebastianburckhardt]

I agree with @sergeybykov that @jkonecki's and @yevhen's descriptions match what I would consider basic ES (without any compiler magic, and without assuming a certain application structure).

Some thoughts of how this could all fit together in Orleans:

• The user defines events explicitly, as classes. As @yevhen says, this is based on application logic, not necessarily on what the grains are. Users can use class hierarchy to organize their events (e.g. AccountEvent is a subclass of BranchEvent is a subclass of BankEvent).
• An event sourced grain (e.g. AccountGrain) specifies (a) methods representing commands, which can read the current grain state and raise events. (b) a class of events generated and consumed by this grain (e.g. AccountEvent) (c) what the grain state is (e.g. int Balance) and how this grain state is updated in response to events (d) which EventJournal to use (several grains can use the same EventJournal)
• The runtime manages the grain state to always be a snapshot of the events in the EventJournal. In particular, grain state cannot be modified in any other way than by raising an event.

Not coincidentally, this sounds a lot like reactive programming and streaming. I think it would be great to implement EventJournals as a special kind of Orleans streams, reusing the current mechanism (perhaps tweaking if necessary). It seems unnecessary to introduce an entirely new extension point for event sourcing providers - I think they are all some (special) kind of stream provider - most likely, a persistent stream provider with some optional extras. For example, it may or may not provide snapshots, aggregation, idempotence.

[sergeybykov]

The elegance of @ReubenBond's approach though is that he doesn't require events defined as classes. His apply() is just a lambda, which is very low overhead to code. On the other hand, explicit event classes may be (I'm speculating) easier for most developers. I wonder if we can get both options somehow in the API.

[yevhen]

This elegance comes with the cost of having absolutely obnoxious projection code. He just not yet discovered that ;)

The law of conservation ...

[richorama]

I also think that replaying messages to a grain is a neat feature. Although this may not be true event sourcing, it could be useful.

-----Original Message----- From: "Yevhen Bobrov" notifications@github.com Sent: ‎20/‎04/‎2015 19:55 To: "dotnet/orleans" orleans@noreply.github.com Cc: "Richard Astbury" richard.astbury@gmail.com Subject: Re: [orleans] Define a programming model/API for Event Sourcing(#343)

This elegance comes with the cost of having absolutely obnoxious projection code. He just not yet discovered that ;) The law of conservation ... — Reply to this email directly or view it on GitHub.

[sebastianburckhardt]

I agree with @richorama that there are definitely scenarios where replaying commands is a neat feature. But I think it is indeed a separate story.

To clarify, suppose that we distinguish command / event terminology as follows:

• commands correspond to the grain methods. They are sent to the grain from the front end, or from some other grain.
• events are raised within commands and sent to the persistence backend. They are classes, not grain methods.

For very simple scenarios, each command just raises an event of the same name, in which case commands and events look like the same thing - and for that special case, it would be perhaps easier to automatically generate the events from the commands. But in general, a command may raise any number of events, breaking this one-to-one correspondence. Also, a command may contain some validation and checking code, which should NOT be part of the event.