Automatic retries

[joffreynorsys]

Hello,

I don't know if it's a good location for a discussion.

Your project is very interesting, it is open source, free, there is a good design and there is a SQL server transport implementation. So i begin to see to use it for a complex application designed with CQRS.

I have a question concerning automatic retries. Shuttle-esb has a such feature ? If yes, how work it ? I test it with a throw exception in the message handler but i don't see retrying.

Another question, you are alone to maintain this project ?

Thank you.

[eben-roux]

Hello,

This is as good a place for a discussion as any, I think :)

Shuttle.Esb does have automatic retries. Any exception in a message handler would result in the message being re-queued. The exception is added to the FailureMessages of the message envelope and the message is either returned to the inbox in the event that there is no deferred queue. If there is a deferred queue (recommended) the message will be enqueued there. The IgnoreTillDate property on the message is set using a back-off strategy. The duration to wait on failure is configurable.

The deferred queue therefore serves a dual purpose. One for deferred messages that only need to be processed a bit later (not for scheduling --- one can use a scheduler for that) and the other as a temporary store for failed messages since they too are only due for processing later. Once messages become due they are moved back to the inbox for the next round.

I am currently the only developer maintaining the project.

Regards, Eben

[joffreynorsys]

Thank you for your response.

Indeed, I see the sequenceId of my message change when i throw an exception. But i thing i do a bad thing...

My handler code write a line in the debug console. With the debugger i place the cursor on the throw exception the first time then i continue the execution (my breakpoint is defined on the Debug.WriteLine line).

 Debug.WriteLine("Simulate error for order id "+context.Message.OrderId);
  int i = 5;
  if (i == 4)
  {
      throw new Exception("error");
  }

If i refresh the content of the sql queue, i see the sequenceId grow up very much (i think the system de-queued and re-queued many times) but my breakpoint is not called

[eben-roux]

Try adding a deferred queue:

<inbox
  workQueueUri="sql://connection/inbox-work"
  deferredQueueUri="sql://connection/inbox-work-deferred"
  errorQueueUri="sql://connection/shuttle-error" />

If you don't use a deferred queue the inbox is going to be thrashed since the message will be enqueued again and then dequeued again and ignored --- it is a vicious cycle :)

During that cycle the handler is not called. Only once the message is due for processing again will the handler be called.

The full configuration can be found here: http://shuttle.github.io/shuttle-esb/full-configuration/

[joffreynorsys]

Ok i understand. There is a configuration for ignore the message during a period. it's work fine with a deferred queue. Thank you.

What do you think about use shuttle to manage message for a cqrs project and specially for build a read model from events ? For me for build a read model from events, the order of messages is very important and i must add a layer for keep the good order...

[eben-roux]

You should never rely on message ordering as that makes your life quite painful :)

However, event ordering in event sourcing is important since the past really only happens in one sequence of events. You can have a look at my ProcessManagement sample in the samples repository.

It makes use of my Shuttle.Recall event sourcing mechanism to store process state (but also demonstrates using custom storing).

I tend to view the following messages as distinct:

• Domain Event / Event Sourcing
• System Messages (Shuttle.Esb)

System messages tend to carry more information than the state required to re-hydrate an aggregate from domain events.

Shuttle messages should not be used as your event sourced messages.

[joffreynorsys]

I agree with you, it's important to differentiate Domain Event Message from System Message.

I think my question is not clear. Let me give you more explaination :

For me, event sourcing is a way to persist your data. it's a persistence layer. using event sourcing as persistence layer is a choice. it's possible to use classic way to store data with CQRS. The command handler change the domain, persist aggregate and publish events on a bus. The persistence and the publishing must be done on double phase commit. So the event sourcing concept et the domain event concept are different.

In my project i use SQL Sever relational table (no event sourcing) to store my datas. I publish the domain event synchronously (in the same transaction scope) for specific needs. Now i need to publish them asynchronously for another need.

So my aim is to find a way to transfert my domain events to the subscriber for update their read model (I looked to NServiceBus wich is very well but it has a commercial licence. Mass transit do not propose SQLServer transport (it's a prerequisite for my project)). And i find your project

As you said, domain event is a message with a particularity : the subscriber end point must be process domain events in the good order.

The purpose of a service bus (like shuttle, nservicebus, masstransit...) is to transfert a message from an endpoint to another(s) endpoint(s). For me, it's a technical layer for the transport. So in this context, the order is not signifiant.

So there is a need to add a layer to manage the order of domain event processing. If i understand, you make that in your samples ?

When i write these lines i begin to think it's very complex to process asynchronously domain events... May be, there is a need to create a specific transport (an event bus ?)...

[eben-roux]

I don't believe that a subscriber should ever be interested in a domain event explicitly. It is possible, but not with a guaranteed delivery order.

When processing domain events is what you need you may want to use projections. I have implemented projections in Shuttle.Recall also. The idea behind it is that every domain event is stored in the EventStore. You can process that store from event 1 at any time --- there is a unique SequenceNumber that is global in that it increments across all aggregates. There can be any number of projections and each start off at position 0.

You can then create a new projection 'X' and add handlers to it to process relevant events.

Each projection runs on a single thread since events cannot be processed in parallel as they are, by their very nature, sequential.

Does this makes sense or am I still missing you :)

[joffreynorsys]

You suggest me to use an event store like a bus for the domain events and use the projection mechanism to build the read model. It's a good idea. I will see it.

Thank you for your responses an your time !

If i use Shuttle, i think i come back here for others questions ;).

[eben-roux]

Please do ask any questions you may have at any time :)