samples

[kiquenet]

Any final solution with full source code sample application ? IMHO, better samples for minimize learning curve are real applications with full source code and good patterns.

[eben-roux]

Agreed. Finding the time for this is somewhat of a problem, though :)

I'll definitely get around to creating something that will demonstrate a full application that includes a process manager.

However, to a large extent the messaging portion of any solution is only a subset of the final product. There are going to be many opinions around front-end, domain, and data storage.

Front-end options:

• Windows
  • WinForms
  • WPF
• Web
  • ASP.NET WebForms
  • ASP.NET MVC
  • Client-side MVC (Ember, Angular, Knockout)

For the domain it is probably a bit easier as just about anything would do.

Data storage:

• CQRS
• Event Sourcing
• Document DB

Also need something that is going to be simple enough to not take too long to develop but complex enough to demonstrate the key elements.

[kiquenet]

I think easy sample like this:

Console application in Client to send message and read message from/to private queue in Server.

Console application in Server to send message and read message from/toprivate queue.

Client send Message A to private Queue in Server

Server read Message A from private queue

Server sendMessage B to private queue

Client read Message B from private queue.

Console Application.exe as Console and too can be installed as Windows Service.

For easy testing, better console application.

[eben-roux]

I cover most of those in the samples: https://github.com/Shuttle/shuttle-esb-samples

Do you reckon I should expand those a bit?

However, I still want to have a more substantial application as a reference.

[mpaul31]

eben

is this project still kicking and being supported? we are getting ready to kick off a new project and i would love to use the project.

[mpaul31]

also, kind of off topic but do you mind sharing how you organize your solution (messages, repos, app services, etc.)?

[eben-roux]

The project certainly is still active. I hope that the core will be stable enough to rarely need attention

On a high level you’ll need:

‘Messages’ in some re-usable assembly for the sender (client) and the receiver (host).

‘Server’ that handles the messages.

The sender could be just about anything from a console application to a web-application to a rest-entry point.

So if we have something like an order bounded context (a la DDD) and our company is called Shuttle we could have:

Shuttle.Order.Messages --- containing RegisterOrderCommand and perhaps an OrderRegisteredEvent. Shuttle.Order.Server --- containing the endpoint that handles the RegisterOrderCommand and then publishes the OrderRegisteredEvent.

The command could come from our Shuttle.Shopping.Mvc web front-end or even our Shuttle.Shopping.Web rest-api interface.

Hope that helps.

[mpaul31]

Perfect that makes sense. Just out of curiosity lets say u have a create something command that does nothing more than add a record to the database. For consistency do u still send a command for example from your mvc app or just write directly to the table? Also do u typically have one server per bounded context?

Sent from my iPhone

On Apr 23, 2015, at 12:26 PM, Eben notifications@github.com wrote:

Hi,

The project certainly is still active. I hope that the core will be stable enough to rarely need attention

On a high level you’ll need:

‘Messages’ in some re-usable assembly for the sender (client) and the receiver (host).

‘Server’ that handles the messages.

The sender could be just about anything from a console application to a web-application to a rest-entry point.

So if we have something like an order bounded context (a la DDD) and our company is called Shuttle we could have:

Shuttle.Order.Messages --- containing RegisterOrderCommand and perhaps an OrderRegisteredEvent. Shuttle.Order.Server --- containing the endpoint that handles the RegisterOrderCommand and then publishes the OrderRegisteredEvent.

The command could come from our Shuttle.Shopping.Mvc web front-end or even our Shuttle.Shopping.Web rest-api interface.

Hope that helps.

Regards, Eben

From: mpaul31 Sent: Wednesday, April 22, 2015 9:12 PM To: Shuttle/shuttle-esb Cc: Eben Subject: SPAM-LOW: Re: [shuttle-esb] samples (#45)

also, kind of off topic but do you mind sharing how you organize your solution (messages, repos, app services, etc.)?

— Reply to this email directly or view it on GitHub. — Reply to this email directly or view it on GitHub.

[eben-roux]

Hello,

Anything going over a service bus results in eventual consistency. If you need 100% consistency at any stage you’ll directly interact with the data store.

Yes, there should be one logical endpoint per bounded context. It really is going to depend on what feels correct. A process manager context may even come into play. I use “logical” since physically you may install workers and use the distribution mechanism to get scalability. Also, an endpoint may process more than one BC --- that would be strange but wouldn’t necessarily be wrong and it would be OK for smaller systems.

Regards, Eben

[mpaul31]

Thanks again you explain things very well.

Just to clarify after watching your idempotence video if I am using msmq, sql server and the idempotence service I don't need dtc correct? Or are there edge cases I still need to use a trans scope around the database insert and sending commands / events?

Sent from my iPhone

On Apr 23, 2015, at 2:20 PM, Eben notifications@github.com wrote:

Hello,

Anything going over a service bus results in eventual consistency. If you need 100% consistency at any stage you’ll directly interact with the data store.

Yes, there should be one logical endpoint per bounded context. It really is going to depend on what feels correct. A process manager context may even come into play. I use “logical” since physically you may install workers and use the distribution mechanism to get scalability. Also, an endpoint may process more than one BC --- that would be strange but wouldn’t necessarily be wrong and it would be OK for smaller systems.

Regards, Eben

From: mpaul31 Sent: Thursday, April 23, 2015 6:45 PM To: Shuttle/shuttle-esb Cc: Eben Subject: SPAM-LOW: Re: [shuttle-esb] samples (#45)

Perfect that makes sense. Just out of curiosity lets say u have a create something command that does nothing more than add a record to the database. For consistency do u still send a command for example from your mvc app or just write directly to the table? Also do u typically have one server per bounded context?

Sent from my iPhone

On Apr 23, 2015, at 12:26 PM, Eben notifications@github.com wrote:

Hi,

The project certainly is still active. I hope that the core will be stable enough to rarely need attention

On a high level you’ll need:

‘Messages’ in some re-usable assembly for the sender (client) and the receiver (host).

‘Server’ that handles the messages.

The sender could be just about anything from a console application to a web-application to a rest-entry point.

So if we have something like an order bounded context (a la DDD) and our company is called Shuttle we could have:

Shuttle.Order.Messages --- containing RegisterOrderCommand and perhaps an OrderRegisteredEvent. Shuttle.Order.Server --- containing the endpoint that handles the RegisterOrderCommand and then publishes the OrderRegisteredEvent.

The command could come from our Shuttle.Shopping.Mvc web front-end or even our Shuttle.Shopping.Web rest-api interface.

Hope that helps.

Regards, Eben

From: mpaul31 Sent: Wednesday, April 22, 2015 9:12 PM To: Shuttle/shuttle-esb Cc: Eben Subject: SPAM-LOW: Re: [shuttle-esb] samples (#45)

also, kind of off topic but do you mind sharing how you organize your solution (messages, repos, app services, etc.)?

— Reply to this email directly or view it on GitHub. — Reply to this email directly or view it on GitHub.

— Reply to this email directly or view it on GitHub. — Reply to this email directly or view it on GitHub.

[eben-roux]

All DTC reliance has been removed. All IQueue implementations are expected to work on an at-least-once delivery mechanism. The idempotence service will take care of removing messages with the same message id --- so 100% guaranteed to be duplicate messages.

[mpaul31]

So how is the situation handled when a database record is inserted but publishing an event fails due to for example the msmq server being down?

Sent from my iPhone

On Apr 24, 2015, at 12:00 AM, Eben notifications@github.com wrote:

Hello,

All DTC reliance has been removed. All IQueue implementations are expected to work on an at-least-once delivery mechanism. The idempotence service will take care of removing messages with the same message id --- so 100% guaranteed to be duplicate messages.

Regards, Eben

-------- Original Message --------

From: "mpaul31" notifications@github.com Sent: Friday, April 24, 2015 2:43 AM To: "Shuttle/shuttle-esb" shuttle-esb@noreply.github.com Subject: SPAM-LOW: Re: [shuttle-esb] samples (#45)

Thanks again you explain things very well.

Just to clarify after watching your idempotence video if I am using msmq, sql server and the idempotence service I don't need dtc correct? Or are there edge cases I still need to use a trans scope around the database insert and sending commands / events?

Sent from my iPhone

On Apr 23, 2015, at 2:20 PM, Eben notifications@github.com wrote:

Hello,

Anything going over a service bus results in eventual consistency. If you need 100% consistency at any stage you'll directly interact with the data store.

Yes, there should be one logical endpoint per bounded context. It really is going to depend on what feels correct. A process manager context may even come into play. I use "logical" since physically you may install workers and use the distribution mechanism to get scalability. Also, an endpoint may process more than one BC --- that would be strange but wouldn't necessarily be wrong and it would be OK for smaller systems.

Regards, Eben

From: mpaul31 Sent: Thursday, April 23, 2015 6:45 PM To: Shuttle/shuttle-esb Cc: Eben Subject: SPAM-LOW: Re: [shuttle-esb] samples (#45)

Perfect that makes sense. Just out of curiosity lets say u have a create something command that does nothing more than add a record to the database. For consistency do u still send a command for example from your mvc app or just write directly to the table? Also do u typically have one server per bounded context?

Sent from my iPhone

On Apr 23, 2015, at 12:26 PM, Eben notifications@github.com wrote:

Hi,

The project certainly is still active. I hope that the core will be stable enough to rarely need attention

On a high level you'll need:

'Messages' in some re-usable assembly for the sender (client) and the receiver (host).

'Server' that handles the messages.

The sender could be just about anything from a console application to a web-application to a rest-entry point.

So if we have something like an order bounded context (a la DDD) and our company is called Shuttle we could have:

Shuttle.Order.Messages --- containing RegisterOrderCommand and perhaps an OrderRegisteredEvent. Shuttle.Order.Server --- containing the endpoint that handles the RegisterOrderCommand and then publishes the OrderRegisteredEvent.

The command could come from our Shuttle.Shopping.Mvc web front-end or even our Shuttle.Shopping.Web rest-api interface.

Hope that helps.

Regards, Eben

From: mpaul31 Sent: Wednesday, April 22, 2015 9:12 PM To: Shuttle/shuttle-esb Cc: Eben Subject: SPAM-LOW: Re: [shuttle-esb] samples (#45)

also, kind of off topic but do you mind sharing how you organize your solution (messages, repos, app services, etc.)?

Reply to this email directly or view it on GitHub.

Reply to this email directly or view it on GitHub.

Reply to this email directly or view it on GitHub.

Reply to this email directly or view it on GitHub.

---

Reply to this email directly or view it on GitHub: https://github.com/Shuttle/shuttle-esb/issues/45#issuecomment-95763610

— Reply to this email directly or view it on GitHub.

[eben-roux]

Hi,

The idempotence service has the responsibility to store sent messages and only delete a message once it has been successfully sent.

So, should the message handling succeed your record is inserted (in a transaction --- if it is a transactional endpoint) and the sent messages are also inserted in the idempotence db (in the same transaction).

Now, the sending fails meaning that the message will be returned to the queue --- as a result of the exception.

The message is then picked up again for processing. The idempotence service will then see that the message handling has succeeded since it has been marked as such in the DB. The inbox pipeline will then skip the messaging handling code and move on to the next bit, being the sending of the stored messages. As each message is sucessfully sent it is then removed from the idempotence DB. In essence it is a DB queue per message and as each message is sent it is ACKed by being deleted.

In this way the messages sent during the message handling event will be guaranteed to be sent eventually.

Hope that makes sense.

Regards, Eben

-------- Original Message --------

From: "mpaul31" notifications@github.com Sent: Friday, April 24, 2015 1:04 PM To: "Shuttle/shuttle-esb" shuttle-esb@noreply.github.com Subject: SPAM-LOW: Re: [shuttle-esb] samples (#45)

So how is the situation handled when a database record is inserted but publishing an event fails due to for example the msmq server being down?

Sent from my iPhone

On Apr 24, 2015, at 12:00 AM, Eben notifications@github.com wrote:

Hello,

All DTC reliance has been removed. All IQueue implementations are expected to work on an at-least-once delivery mechanism. The idempotence service will take care of removing messages with the same message id --- so 100% guaranteed to be duplicate messages.

Regards, Eben

-------- Original Message --------

From: "mpaul31" notifications@github.com Sent: Friday, April 24, 2015 2:43 AM To: "Shuttle/shuttle-esb" shuttle-esb@noreply.github.com Subject: SPAM-LOW: Re: [shuttle-esb] samples (#45)

Thanks again you explain things very well.

Just to clarify after watching your idempotence video if I am using msmq, sql server and the idempotence service I don't need dtc correct? Or are there edge cases I still need to use a trans scope around the database insert and sending commands / events?

Sent from my iPhone

On Apr 23, 2015, at 2:20 PM, Eben notifications@github.com wrote:

Hello,

Anything going over a service bus results in eventual consistency. If you need 100% consistency at any stage you'll directly interact with the data store.

Yes, there should be one logical endpoint per bounded context. It really is going to depend on what feels correct. A process manager context may even come into play. I use "logical" since physically you may install workers and use the distribution mechanism to get scalability. Also, an endpoint may process more than one BC --- that would be strange but wouldn't necessarily be wrong and it would be OK for smaller systems.

Regards, Eben

From: mpaul31 Sent: Thursday, April 23, 2015 6:45 PM To: Shuttle/shuttle-esb Cc: Eben Subject: SPAM-LOW: Re: [shuttle-esb] samples (#45)

Perfect that makes sense. Just out of curiosity lets say u have a create something command that does nothing more than add a record to the database. For consistency do u still send a command for example from your mvc app or just write directly to the table? Also do u typically have one server per bounded context?

Sent from my iPhone

On Apr 23, 2015, at 12:26 PM, Eben notifications@github.com wrote:

Hi,

The project certainly is still active. I hope that the core will be stable enough to rarely need attention

On a high level you'll need:

'Messages' in some re-usable assembly for the sender (client) and the receiver (host).

'Server' that handles the messages.

The sender could be just about anything from a console application to a web-application to a rest-entry point.

So if we have something like an order bounded context (a la DDD) and our company is called Shuttle we could have:

Shuttle.Order.Messages --- containing RegisterOrderCommand and perhaps an OrderRegisteredEvent. Shuttle.Order.Server --- containing the endpoint that handles the RegisterOrderCommand and then publishes the OrderRegisteredEvent.

The command could come from our Shuttle.Shopping.Mvc web front-end or even our Shuttle.Shopping.Web rest-api interface.

Hope that helps.

Regards, Eben

From: mpaul31 Sent: Wednesday, April 22, 2015 9:12 PM To: Shuttle/shuttle-esb Cc: Eben Subject: SPAM-LOW: Re: [shuttle-esb] samples (#45)

also, kind of off topic but do you mind sharing how you organize your solution (messages, repos, app services, etc.)?

Reply to this email directly or view it on GitHub.

Reply to this email directly or view it on GitHub.

Reply to this email directly or view it on GitHub.

Reply to this email directly or view it on GitHub.

---

Reply to this email directly or view it on GitHub: https://github.com/Shuttle/shuttle-esb/issues/45#issuecomment-95763610

Reply to this email directly or view it on GitHub.

---

Reply to this email directly or view it on GitHub: https://github.com/Shuttle/shuttle-esb/issues/45#issuecomment-95896543

[mpaul31]

great thank again eben! just wanted to make sure because we are building a financial application.

On Fri, Apr 24, 2015 at 8:32 AM, Eben notifications@github.com wrote:

Hi,

The idempotence service has the responsibility to store sent messages and only delete a message once it has been successfully sent.

So, should the message handling succeed your record is inserted (in a transaction --- if it is a transactional endpoint) and the sent messages are also inserted in the idempotence db (in the same transaction).

Now, the sending fails meaning that the message will be returned to the queue --- as a result of the exception.

The message is then picked up again for processing. The idempotence service will then see that the message handling has succeeded since it has been marked as such in the DB. The inbox pipeline will then skip the messaging handling code and move on to the next bit, being the sending of the stored messages. As each message is sucessfully sent it is then removed from the idempotence DB. In essence it is a DB queue per message and as each message is sent it is ACKed by being deleted.

In this way the messages sent during the message handling event will be guaranteed to be sent eventually.

Hope that makes sense.

Regards, Eben

-------- Original Message --------

From: "mpaul31" notifications@github.com Sent: Friday, April 24, 2015 1:04 PM To: "Shuttle/shuttle-esb" shuttle-esb@noreply.github.com Subject: SPAM-LOW: Re: [shuttle-esb] samples (#45)

So how is the situation handled when a database record is inserted but publishing an event fails due to for example the msmq server being down?

Sent from my iPhone

On Apr 24, 2015, at 12:00 AM, Eben notifications@github.com wrote:

Hello,

All DTC reliance has been removed. All IQueue implementations are expected to work on an at-least-once delivery mechanism. The idempotence service will take care of removing messages with the same message id --- so 100% guaranteed to be duplicate messages.

Regards, Eben

-------- Original Message --------

From: "mpaul31" notifications@github.com Sent: Friday, April 24, 2015 2:43 AM To: "Shuttle/shuttle-esb" shuttle-esb@noreply.github.com Subject: SPAM-LOW: Re: [shuttle-esb] samples (#45)

Thanks again you explain things very well.

Just to clarify after watching your idempotence video if I am using msmq, sql server and the idempotence service I don't need dtc correct? Or are there edge cases I still need to use a trans scope around the database insert and sending commands / events?

Sent from my iPhone

On Apr 23, 2015, at 2:20 PM, Eben notifications@github.com wrote:

Hello,

Anything going over a service bus results in eventual consistency. If you need 100% consistency at any stage you'll directly interact with the data store.

Yes, there should be one logical endpoint per bounded context. It really is going to depend on what feels correct. A process manager context may even come into play. I use "logical" since physically you may install workers and use the distribution mechanism to get scalability. Also, an endpoint may process more than one BC --- that would be strange but wouldn't necessarily be wrong and it would be OK for smaller systems.

Regards, Eben

From: mpaul31 Sent: Thursday, April 23, 2015 6:45 PM To: Shuttle/shuttle-esb Cc: Eben Subject: SPAM-LOW: Re: [shuttle-esb] samples (#45)

Perfect that makes sense. Just out of curiosity lets say u have a create something command that does nothing more than add a record to the database. For consistency do u still send a command for example from your mvc app or just write directly to the table? Also do u typically have one server per bounded context?

Sent from my iPhone

On Apr 23, 2015, at 12:26 PM, Eben notifications@github.com wrote:

Hi,

The project certainly is still active. I hope that the core will be stable enough to rarely need attention

On a high level you'll need:

'Messages' in some re-usable assembly for the sender (client) and the receiver (host).

'Server' that handles the messages.

The sender could be just about anything from a console application to a web-application to a rest-entry point.

So if we have something like an order bounded context (a la DDD) and our company is called Shuttle we could have:

Shuttle.Order.Messages --- containing RegisterOrderCommand and perhaps an OrderRegisteredEvent. Shuttle.Order.Server --- containing the endpoint that handles the RegisterOrderCommand and then publishes the OrderRegisteredEvent.

The command could come from our Shuttle.Shopping.Mvc web front-end or even our Shuttle.Shopping.Web rest-api interface.

Hope that helps.

Regards, Eben

From: mpaul31 Sent: Wednesday, April 22, 2015 9:12 PM To: Shuttle/shuttle-esb Cc: Eben Subject: SPAM-LOW: Re: [shuttle-esb] samples (#45)

also, kind of off topic but do you mind sharing how you organize your solution (messages, repos, app services, etc.)?

Reply to this email directly or view it on GitHub.

Reply to this email directly or view it on GitHub.

Reply to this email directly or view it on GitHub.

Reply to this email directly or view it on GitHub.

---

Reply to this email directly or view it on GitHub:

https://github.com/Shuttle/shuttle-esb/issues/45#issuecomment-95763610

Reply to this email directly or view it on GitHub.

---

Reply to this email directly or view it on GitHub: https://github.com/Shuttle/shuttle-esb/issues/45#issuecomment-95896543

— Reply to this email directly or view it on GitHub https://github.com/Shuttle/shuttle-esb/issues/45#issuecomment-95918329.

[mpaul31]

eben,

sorry for cluttering up this thread so let me know if you want me to form my questions somewhere else.

i was looking at the shuttle-email and i noticed it uses the ActiveTimeRange module. I don't quite understand the intent of this module. Could you please elaborate a little bit?

Thanks,

On Fri, Apr 24, 2015 at 9:07 AM, Marco Paul mpaul31@gmail.com wrote:

great thank again eben! just wanted to make sure because we are building a financial application.

On Fri, Apr 24, 2015 at 8:32 AM, Eben notifications@github.com wrote:

Hi,

The idempotence service has the responsibility to store sent messages and only delete a message once it has been successfully sent.

So, should the message handling succeed your record is inserted (in a transaction --- if it is a transactional endpoint) and the sent messages are also inserted in the idempotence db (in the same transaction).

Now, the sending fails meaning that the message will be returned to the queue --- as a result of the exception.

The message is then picked up again for processing. The idempotence service will then see that the message handling has succeeded since it has been marked as such in the DB. The inbox pipeline will then skip the messaging handling code and move on to the next bit, being the sending of the stored messages. As each message is sucessfully sent it is then removed from the idempotence DB. In essence it is a DB queue per message and as each message is sent it is ACKed by being deleted.

In this way the messages sent during the message handling event will be guaranteed to be sent eventually.

Hope that makes sense.

Regards, Eben

-------- Original Message --------

From: "mpaul31" notifications@github.com Sent: Friday, April 24, 2015 1:04 PM To: "Shuttle/shuttle-esb" shuttle-esb@noreply.github.com Subject: SPAM-LOW: Re: [shuttle-esb] samples (#45)

So how is the situation handled when a database record is inserted but publishing an event fails due to for example the msmq server being down?

Sent from my iPhone

On Apr 24, 2015, at 12:00 AM, Eben notifications@github.com wrote:

Hello,

All DTC reliance has been removed. All IQueue implementations are expected to work on an at-least-once delivery mechanism. The idempotence service will take care of removing messages with the same message id --- so 100% guaranteed to be duplicate messages.

Regards, Eben

-------- Original Message --------

From: "mpaul31" notifications@github.com Sent: Friday, April 24, 2015 2:43 AM To: "Shuttle/shuttle-esb" shuttle-esb@noreply.github.com Subject: SPAM-LOW: Re: [shuttle-esb] samples (#45)

Thanks again you explain things very well.

Just to clarify after watching your idempotence video if I am using msmq, sql server and the idempotence service I don't need dtc correct? Or are there edge cases I still need to use a trans scope around the database insert and sending commands / events?

Sent from my iPhone

On Apr 23, 2015, at 2:20 PM, Eben notifications@github.com wrote:

Hello,

Anything going over a service bus results in eventual consistency. If you need 100% consistency at any stage you'll directly interact with the data store.

Yes, there should be one logical endpoint per bounded context. It really is going to depend on what feels correct. A process manager context may even come into play. I use "logical" since physically you may install workers and use the distribution mechanism to get scalability. Also, an endpoint may process more than one BC --- that would be strange but wouldn't necessarily be wrong and it would be OK for smaller systems.

Regards, Eben

From: mpaul31 Sent: Thursday, April 23, 2015 6:45 PM To: Shuttle/shuttle-esb Cc: Eben Subject: SPAM-LOW: Re: [shuttle-esb] samples (#45)

Perfect that makes sense. Just out of curiosity lets say u have a create something command that does nothing more than add a record to the database. For consistency do u still send a command for example from your mvc app or just write directly to the table? Also do u typically have one server per bounded context?

Sent from my iPhone

On Apr 23, 2015, at 12:26 PM, Eben notifications@github.com wrote:

Hi,

The project certainly is still active. I hope that the core will be stable enough to rarely need attention

On a high level you'll need:

'Messages' in some re-usable assembly for the sender (client) and the receiver (host).

'Server' that handles the messages.

The sender could be just about anything from a console application to a web-application to a rest-entry point.

So if we have something like an order bounded context (a la DDD) and our company is called Shuttle we could have:

Shuttle.Order.Messages --- containing RegisterOrderCommand and perhaps an OrderRegisteredEvent. Shuttle.Order.Server --- containing the endpoint that handles the RegisterOrderCommand and then publishes the OrderRegisteredEvent.

The command could come from our Shuttle.Shopping.Mvc web front-end or even our Shuttle.Shopping.Web rest-api interface.

Hope that helps.

Regards, Eben

From: mpaul31 Sent: Wednesday, April 22, 2015 9:12 PM To: Shuttle/shuttle-esb Cc: Eben Subject: SPAM-LOW: Re: [shuttle-esb] samples (#45)

also, kind of off topic but do you mind sharing how you organize your solution (messages, repos, app services, etc.)?

Reply to this email directly or view it on GitHub.

Reply to this email directly or view it on GitHub.

Reply to this email directly or view it on GitHub.

Reply to this email directly or view it on GitHub.

---

Reply to this email directly or view it on GitHub:

https://github.com/Shuttle/shuttle-esb/issues/45#issuecomment-95763610

Reply to this email directly or view it on GitHub.

---

Reply to this email directly or view it on GitHub: https://github.com/Shuttle/shuttle-esb/issues/45#issuecomment-95896543

— Reply to this email directly or view it on GitHub https://github.com/Shuttle/shuttle-esb/issues/45#issuecomment-95918329.

[eben-roux]

Hi,

Absolutely no problem.

The modules in shuttle allow one to add functionality to the various pipelines. So what the ActiveTimeRange module does is abort a pipeline on startup should the current time not fall within the active time range.

I initially wrote the module since one of the systems that shuttle was used in at a client site had to stop functioning around 20h00 at night and start up again around 7h00 in the morning. The particular client used VM snapshots to back up their systems and for this they needed to stop various services that the shuttle components relied upon. This shutdown business happened around 20h30 and the startups again around 6h00. So the ActiveTimeRange simply ensured that shuttle would not end up with messages in the error queue

So if you need the shuttle endpoint up all the time then there is no need for the ActiveTimeRange module.

Regards, Eben

[mpaul31]

ok thank you that makes sense.

we plan on using msmq as our transport. i am thinking we could setup a message distributor on the web server the client runs on and run our shuttle servers on one of more app servers. this way if the msmq server is down for some reason most likely the website is down as well. how does this sound? how do u typically handle msmq setup/distribution?

On Sat, Apr 25, 2015 at 2:03 AM, Eben notifications@github.com wrote:

Hi,

Absolutely no problem.

The modules in shuttle allow one to add functionality to the various pipelines. So what the ActiveTimeRange module does is abort a pipeline on startup should the current time not fall within the active time range.

I initially wrote the module since one of the systems that shuttle was used in at a client site had to stop functioning around 20h00 at night and start up again around 7h00 in the morning. The particular client used VM snapshots to back up their systems and for this they needed to stop various services that the shuttle components relied upon. This shutdown business happened around 20h30 and the startups again around 6h00. So the ActiveTimeRange simply ensured that shuttle would not end up with messages in the error queue

So if you need the shuttle endpoint up all the time then there is no need for the ActiveTimeRange module.

Regards, Eben

From: mpaul31 Sent: Friday, April 24, 2015 10:18 PM

To: Shuttle/shuttle-esb Cc: Eben Subject: SPAM-LOW: Re: [shuttle-esb] samples (#45)

eben,

sorry for cluttering up this thread so let me know if you want me to form my questions somewhere else.

i was looking at the shuttle-email and i noticed it uses the ActiveTimeRange module. I don't quite understand the intent of this module. Could you please elaborate a little bit?

Thanks,

On Fri, Apr 24, 2015 at 9:07 AM, Marco Paul mpaul31@gmail.com wrote:

great thank again eben! just wanted to make sure because we are building a financial application.

On Fri, Apr 24, 2015 at 8:32 AM, Eben notifications@github.com wrote:

Hi,

The idempotence service has the responsibility to store sent messages and only delete a message once it has been successfully sent.

So, should the message handling succeed your record is inserted (in a transaction --- if it is a transactional endpoint) and the sent messages are also inserted in the idempotence db (in the same transaction).

Now, the sending fails meaning that the message will be returned to the queue --- as a result of the exception.

The message is then picked up again for processing. The idempotence service will then see that the message handling has succeeded since it has been marked as such in the DB. The inbox pipeline will then skip the messaging handling code and move on to the next bit, being the sending of the stored messages. As each message is sucessfully sent it is then removed from the idempotence DB. In essence it is a DB queue per message and as each message is sent it is ACKed by being deleted.

In this way the messages sent during the message handling event will be guaranteed to be sent eventually.

Hope that makes sense.

Regards, Eben

-------- Original Message --------

From: "mpaul31" notifications@github.com Sent: Friday, April 24, 2015 1:04 PM To: "Shuttle/shuttle-esb" shuttle-esb@noreply.github.com Subject: SPAM-LOW: Re: [shuttle-esb] samples (#45)

So how is the situation handled when a database record is inserted but publishing an event fails due to for example the msmq server being down?

Sent from my iPhone

On Apr 24, 2015, at 12:00 AM, Eben notifications@github.com wrote:

Hello,

All DTC reliance has been removed. All IQueue implementations are expected to work on an at-least-once delivery mechanism. The idempotence service will take care of removing messages with the same message id --- so 100% guaranteed to be duplicate messages.

Regards, Eben

-------- Original Message --------

From: "mpaul31" notifications@github.com Sent: Friday, April 24, 2015 2:43 AM To: "Shuttle/shuttle-esb" shuttle-esb@noreply.github.com Subject: SPAM-LOW: Re: [shuttle-esb] samples (#45)

Thanks again you explain things very well.

Just to clarify after watching your idempotence video if I am using msmq, sql server and the idempotence service I don't need dtc correct? Or are there edge cases I still need to use a trans scope around the database insert and sending commands / events?

Sent from my iPhone

On Apr 23, 2015, at 2:20 PM, Eben notifications@github.com wrote:

Hello,

Anything going over a service bus results in eventual consistency. If you need 100% consistency at any stage you'll directly interact with the data store.

Yes, there should be one logical endpoint per bounded context. It really is going to depend on what feels correct. A process manager context may even come into play. I use "logical" since physically you may install workers and use the distribution mechanism to get scalability. Also, an endpoint may process more than one BC --- that would be strange but wouldn't necessarily be wrong and it would be OK for smaller systems.

Regards, Eben

From: mpaul31 Sent: Thursday, April 23, 2015 6:45 PM To: Shuttle/shuttle-esb Cc: Eben Subject: SPAM-LOW: Re: [shuttle-esb] samples (#45)

Perfect that makes sense. Just out of curiosity lets say u have a create something command that does nothing more than add a record to the database. For consistency do u still send a command for example from your mvc app or just write directly to the table? Also do u typically have one server per bounded context?

Sent from my iPhone

On Apr 23, 2015, at 12:26 PM, Eben notifications@github.com wrote:

Hi,

The project certainly is still active. I hope that the core will be stable enough to rarely need attention

On a high level you'll need:

'Messages' in some re-usable assembly for the sender (client) and the receiver (host).

'Server' that handles the messages.

The sender could be just about anything from a console application to a web-application to a rest-entry point.

So if we have something like an order bounded context (a la DDD) and our company is called Shuttle we could have:

Shuttle.Order.Messages --- containing RegisterOrderCommand and perhaps an OrderRegisteredEvent. Shuttle.Order.Server --- containing the endpoint that handles the RegisterOrderCommand and then publishes the OrderRegisteredEvent.

The command could come from our Shuttle.Shopping.Mvc web front-end or even our Shuttle.Shopping.Web rest-api interface.

Hope that helps.

Regards, Eben

From: mpaul31 Sent: Wednesday, April 22, 2015 9:12 PM To: Shuttle/shuttle-esb Cc: Eben Subject: SPAM-LOW: Re: [shuttle-esb] samples (#45)

also, kind of off topic but do you mind sharing how you organize your solution (messages, repos, app services, etc.)?

Reply to this email directly or view it on GitHub.

Reply to this email directly or view it on GitHub.

Reply to this email directly or view it on GitHub.

Reply to this email directly or view it on GitHub.

---

Reply to this email directly or view it on GitHub:

https://github.com/Shuttle/shuttle-esb/issues/45#issuecomment-95763610

Reply to this email directly or view it on GitHub.

---

Reply to this email directly or view it on GitHub:

https://github.com/Shuttle/shuttle-esb/issues/45#issuecomment-95896543

— Reply to this email directly or view it on GitHub <https://github.com/Shuttle/shuttle-esb/issues/45#issuecomment-95918329 .

— Reply to this email directly or view it on GitHub.

— Reply to this email directly or view it on GitHub https://github.com/Shuttle/shuttle-esb/issues/45#issuecomment-96135396.

[mpaul31]

also i just read the following from the "autonomous component" section:

"Even if the entire machine is brought down Shuttle ESB will still store message on the sending machine when using an outbox"

so if using msmq, does this mean all places the send message need msmq installed in the case the destination endpoint/queue is down?

On Mon, Apr 27, 2015 at 8:28 AM, Marco Paul mpaul31@gmail.com wrote:

ok thank you that makes sense.

we plan on using msmq as our transport. i am thinking we could setup a message distributor on the web server the client runs on and run our shuttle servers on one of more app servers. this way if the msmq server is down for some reason most likely the website is down as well. how does this sound? how do u typically handle msmq setup/distribution?

On Sat, Apr 25, 2015 at 2:03 AM, Eben notifications@github.com wrote:

Hi,

Absolutely no problem.

The modules in shuttle allow one to add functionality to the various pipelines. So what the ActiveTimeRange module does is abort a pipeline on startup should the current time not fall within the active time range.

I initially wrote the module since one of the systems that shuttle was used in at a client site had to stop functioning around 20h00 at night and start up again around 7h00 in the morning. The particular client used VM snapshots to back up their systems and for this they needed to stop various services that the shuttle components relied upon. This shutdown business happened around 20h30 and the startups again around 6h00. So the ActiveTimeRange simply ensured that shuttle would not end up with messages in the error queue

So if you need the shuttle endpoint up all the time then there is no need for the ActiveTimeRange module.

Regards, Eben

From: mpaul31 Sent: Friday, April 24, 2015 10:18 PM

To: Shuttle/shuttle-esb Cc: Eben Subject: SPAM-LOW: Re: [shuttle-esb] samples (#45)

eben,

sorry for cluttering up this thread so let me know if you want me to form my questions somewhere else.

i was looking at the shuttle-email and i noticed it uses the ActiveTimeRange module. I don't quite understand the intent of this module. Could you please elaborate a little bit?

Thanks,

On Fri, Apr 24, 2015 at 9:07 AM, Marco Paul mpaul31@gmail.com wrote:

great thank again eben! just wanted to make sure because we are building a financial application.

On Fri, Apr 24, 2015 at 8:32 AM, Eben notifications@github.com wrote:

Hi,

The idempotence service has the responsibility to store sent messages and only delete a message once it has been successfully sent.

So, should the message handling succeed your record is inserted (in a transaction --- if it is a transactional endpoint) and the sent messages are also inserted in the idempotence db (in the same transaction).

Now, the sending fails meaning that the message will be returned to the queue --- as a result of the exception.

The message is then picked up again for processing. The idempotence service will then see that the message handling has succeeded since it has been marked as such in the DB. The inbox pipeline will then skip the messaging handling code and move on to the next bit, being the sending of the stored messages. As each message is sucessfully sent it is then removed from the idempotence DB. In essence it is a DB queue per message and as each message is sent it is ACKed by being deleted.

In this way the messages sent during the message handling event will be guaranteed to be sent eventually.

Hope that makes sense.

Regards, Eben

-------- Original Message --------

From: "mpaul31" notifications@github.com Sent: Friday, April 24, 2015 1:04 PM To: "Shuttle/shuttle-esb" shuttle-esb@noreply.github.com Subject: SPAM-LOW: Re: [shuttle-esb] samples (#45)

So how is the situation handled when a database record is inserted but publishing an event fails due to for example the msmq server being down?

Sent from my iPhone

On Apr 24, 2015, at 12:00 AM, Eben notifications@github.com wrote:

Hello,

All DTC reliance has been removed. All IQueue implementations are expected to work on an at-least-once delivery mechanism. The idempotence service will take care of removing messages with the same message id --- so 100% guaranteed to be duplicate messages.

Regards, Eben

-------- Original Message --------

From: "mpaul31" notifications@github.com Sent: Friday, April 24, 2015 2:43 AM To: "Shuttle/shuttle-esb" shuttle-esb@noreply.github.com Subject: SPAM-LOW: Re: [shuttle-esb] samples (#45)

Thanks again you explain things very well.

Just to clarify after watching your idempotence video if I am using msmq, sql server and the idempotence service I don't need dtc correct? Or are there edge cases I still need to use a trans scope around the database insert and sending commands / events?

Sent from my iPhone

On Apr 23, 2015, at 2:20 PM, Eben notifications@github.com wrote:

Hello,

Anything going over a service bus results in eventual consistency. If you need 100% consistency at any stage you'll directly interact with the data store.

Yes, there should be one logical endpoint per bounded context. It really is going to depend on what feels correct. A process manager context may even come into play. I use "logical" since physically you may install workers and use the distribution mechanism to get scalability. Also, an endpoint may process more than one BC --- that would be strange but wouldn't necessarily be wrong and it would be OK for smaller systems.

Regards, Eben

From: mpaul31 Sent: Thursday, April 23, 2015 6:45 PM To: Shuttle/shuttle-esb Cc: Eben Subject: SPAM-LOW: Re: [shuttle-esb] samples (#45)

Perfect that makes sense. Just out of curiosity lets say u have a create something command that does nothing more than add a record to the database. For consistency do u still send a command for example from your mvc app or just write directly to the table? Also do u typically have one server per bounded context?

Sent from my iPhone

On Apr 23, 2015, at 12:26 PM, Eben <notifications@github.com

wrote:

Hi,

The project certainly is still active. I hope that the core will be stable enough to rarely need attention

On a high level you'll need:

'Messages' in some re-usable assembly for the sender (client) and the receiver (host).

'Server' that handles the messages.

The sender could be just about anything from a console application to a web-application to a rest-entry point.

So if we have something like an order bounded context (a la DDD) and our company is called Shuttle we could have:

Shuttle.Order.Messages --- containing RegisterOrderCommand and perhaps an OrderRegisteredEvent. Shuttle.Order.Server --- containing the endpoint that handles the RegisterOrderCommand and then publishes the OrderRegisteredEvent.

The command could come from our Shuttle.Shopping.Mvc web front-end or even our Shuttle.Shopping.Web rest-api interface.

Hope that helps.

Regards, Eben

From: mpaul31 Sent: Wednesday, April 22, 2015 9:12 PM To: Shuttle/shuttle-esb Cc: Eben Subject: SPAM-LOW: Re: [shuttle-esb] samples (#45)

also, kind of off topic but do you mind sharing how you organize your solution (messages, repos, app services, etc.)?

Reply to this email directly or view it on GitHub.

Reply to this email directly or view it on GitHub.

Reply to this email directly or view it on GitHub.

Reply to this email directly or view it on GitHub.

---

Reply to this email directly or view it on GitHub:

https://github.com/Shuttle/shuttle-esb/issues/45#issuecomment-95763610

Reply to this email directly or view it on GitHub.

---

Reply to this email directly or view it on GitHub:

https://github.com/Shuttle/shuttle-esb/issues/45#issuecomment-95896543

— Reply to this email directly or view it on GitHub < https://github.com/Shuttle/shuttle-esb/issues/45#issuecomment-95918329>.

— Reply to this email directly or view it on GitHub.

— Reply to this email directly or view it on GitHub https://github.com/Shuttle/shuttle-esb/issues/45#issuecomment-96135396.

[eben-roux]

With MSMQ you should only use the local queues. Distribution will be handled by shuttle. There is always a single logical endpoint for a command. This translates to a physical endpoint where distribution can be done. So any endpoint can, effectively, distribute messages but typically it is the main entry endpoint. To set up a worker is a simple process but let me know if you need some more information.

As far as the sender is concerned: if there is no "outbox" / outgoing queue there will probably be an issue. You will not be able to send a message if MSMQ is not installed AFAIK since it makes use of an outgoing queue. It is a rather nifty thing the outgoing queue but if you send to a remote queue that will never exist that outgoing message does stick around. I am sure there will be properties on can set to eventually remove a message that cannot reach its destination but ideally this isn't something that should be a real problem.

For scenarios where you use a broker like RabbitMQ there is no outgoing queue. In such a case one could opt to install RabbitMQ on the local machine and configure an "outbox" in shuttle as a store-and-forward mechanism similar to the MSMQ outgoing queue. Without such as mechanism the send will fail outright should the destination queue not be available. These are architectural choices that you would need to make based on some trade-offs. Since it is so simple to configure it will not affect any real design choices up-front as you are not going to paint yourself into a corner :)

Hope that helps.

p.s. I apologise for the late reply. We had a public holiday in South Africa yesterday and I was out-of-town.

[mpaul31]

ok i have been thinking about this backwards then. So, your sender always sends its messages to the local msmq queue and your server, for example let's say runs on a remote app server, has its workQueueUri inbox point back senders remote queue. am i understanding correctly? so the servers pull from the sender. i was thinking the senders were directly pushing to the remote servers inbox queue.

no worries about the late reply, i was actually worried something happened j/k :)

On Tue, Apr 28, 2015 at 1:13 AM, Eben notifications@github.com wrote:

With MSMQ you should only use the local queues. Distribution will be handled by shuttle. There is always a single logical endpoint for a command. This translates to a physical endpoint where distribution can be done. So any endpoint can, effectively, distribute messages but typically it is the main entry endpoint. To set up a worker is a simple process but let me know if you need some more information.

As far as the sender is concerned: if there is no "outbox" / outgoing queue there will probably be an issue. You will not be able to send a message if MSMQ is not installed AFAIK since it makes use of an outgoing queue. It is a rather nifty thing the outgoing queue but if you send to a remote queue that will never exist that outgoing message does stick around. I am sure there will be properties on can set to eventually remove a message that cannot reach its destination but ideally this isn't something that should be a real problem.

For scenarios where you use a broker like RabbitMQ there is no outgoing queue. In such a case one could opt to install RabbitMQ on the local machine and configure an "outbox" in shuttle as a store-and-forward mechanism similar to the MSMQ outgoing queue. Without such as mechanism the send will fail outright should the destination queue not be available. These are architectural choices that you would need to make based on some trade-offs. Since it is so simple to configure it will not affect any real design choices up-front as you are not going to paint yourself into a corner :)

Hope that helps.

p.s. I apologise for the late reply. We had a public holiday in South Africa yesterday and I was out-of-town.

— Reply to this email directly or view it on GitHub https://github.com/Shuttle/shuttle-esb/issues/45#issuecomment-96911635.

[eben-roux]

Side note: a sender does not necessarily have an inbox so it may be only a sender.

In an MSMQ environment the sender would send to the local queue on the receiver. The "outgoing queue" is an infrastructural dance that MSMQ performs. You typically don't get involved in that.

So the receiver picks up the messages from its local inbox queue uri.

With shuttle you can explicitly define an "outbox" that can be hosted anywhere. However, it makes the most sense to have it on the local box but also only when required. For MSMQ an outbox has no value as MSMQ already has the outgoing queue "behind the scenes".

So your thinking is 100%: the sender does send directly to the recipient's inbox queue but the message is not necessarily immediately transported into that queue. For scenarios where there is no outbox and no outgoing queue implemented by the queuing technology it will be placed into the recipient's queue immediately; failing which an exception occurs.

It can get tricky but I hope that helps.

[mpaul31]

yeah i see how this can get a little confusing. so in my senders app.config file my messageRoute uri would point to the recipients inbox which is located on another machine (remote: msmq://app-server/server_inbox_work), however msmq will first behind the scenes use the "outgoing queue" (in case the remote server is down) and forward when possible. so my server then points to its local inbox. that sound right?

lots of terminology i need to brush on :)

On Tue, Apr 28, 2015 at 8:20 AM, Eben notifications@github.com wrote:

Side note: a sender does not necessarily have an inbox so it may be only a sender.

In an MSMQ environment the sender would send to the local queue on the receiver. The "outgoing queue" is an infrastructural dance that MSMQ performs. You typically don't get involved in that.

So the receiver picks up the messages from its local inbox queue uri.

With shuttle you can explicitly define an "outbox" that can be hosted anywhere. However, it makes the most sense to have it on the local box but also only when required. For MSMQ an outbox has no value as MSMQ already has the outgoing queue "behind the scenes".

So your thinking is 100%: the sender does send directly to the recipient's inbox queue but the message is not necessarily immediately transported into that queue. For scenarios where there is no outbox and no outgoing queue implemented by the queuing technology it will be placed into the recipient's queue immediately; failing which an exception occurs.

It can get tricky but I hope that helps.

— Reply to this email directly or view it on GitHub https://github.com/Shuttle/shuttle-esb/issues/45#issuecomment-97041704.

[eben-roux]

That is correct :)

[mpaul31]

​about damn time! :)

thanks again eben, you are definitely helping the learning curve.​ next up on my list is checking out your administration application to see what functionality it provides.

On Tue, Apr 28, 2015 at 8:46 AM, Eben notifications@github.com wrote:

That is correct :)

— Reply to this email directly or view it on GitHub https://github.com/Shuttle/shuttle-esb/issues/45#issuecomment-97049880.

[mpaul31]

ok i'm trying to follow along with the online sample and i receiving the following error in my client

"No queue factory has been registered for scheme msmq"

Any ideas what i could be missing? I copied all the app.config stuff right from the site.

thanks eben

On Tue, Apr 28, 2015 at 8:51 AM, Marco Paul mpaul31@gmail.com wrote:

​about damn time! :)

thanks again eben, you are definitely helping the learning curve.​ next up on my list is checking out your administration application to see what functionality it provides.

On Tue, Apr 28, 2015 at 8:46 AM, Eben notifications@github.com wrote:

That is correct :)

— Reply to this email directly or view it on GitHub https://github.com/Shuttle/shuttle-esb/issues/45#issuecomment-97049880.

[mpaul31]

nevermind sorry about the last post. when i rolled back my project the msmq nuget got removed.

On Wed, Apr 29, 2015 at 4:13 PM, Marco Paul mpaul31@gmail.com wrote:

ok i'm trying to follow along with the online sample and i receiving the following error in my client

"No queue factory has been registered for scheme msmq"

Any ideas what i could be missing? I copied all the app.config stuff right from the site.

thanks eben

On Tue, Apr 28, 2015 at 8:51 AM, Marco Paul mpaul31@gmail.com wrote:

​about damn time! :)

thanks again eben, you are definitely helping the learning curve.​ next up on my list is checking out your administration application to see what functionality it provides.

On Tue, Apr 28, 2015 at 8:46 AM, Eben notifications@github.com wrote:

That is correct :)

— Reply to this email directly or view it on GitHub https://github.com/Shuttle/shuttle-esb/issues/45#issuecomment-97049880 .

[eben-roux]

No problem.

There are two ways to get those factories registered. The default mechanism scans the application folder for any implementations of the IQueueFactory interface; else you could specify the implementation directly via the config file:

<serviceBus>
    <queueFactories scan="true|false">
        <add type="Shuttle.ESB.Msmq.MsmqQueueFactory, Shuttle.ESB.Msmq" />
        <add type="Shuttle.ESB.RabbitMQ.RabbitMQQueueFactory, Shuttle.ESB.RabbitMQ" />
        <add type="Shuttle.ESB.SqlServer.SqlQueueFactory, Shuttle.ESB.SqlServer" />
    </queueFactories>
</serviceBus>

[mpaul31]

ok so i got some things setup. is there any documentation on what is needed/required when publishing and subscribing? do all the publishers and subscribers need the app.config to point to the sql server subscription connection string? i'm a little confused because looking at the examples the client has a sqlServer config section but the subscribers do not have a config section but do have a connection string settings. just not sure what is required?

thanks!

On Wed, Apr 29, 2015 at 11:43 PM, Eben notifications@github.com wrote:

No problem.

There are two ways to get those factories registered. The default mechanism scans the application folder for any implementations of the IQueueFactory interface; else you could specify the implementation directly via the config file:

— Reply to this email directly or view it on GitHub https://github.com/Shuttle/shuttle-esb/issues/45#issuecomment-97652427.

[eben-roux]

All subscriptions have to be stored in some persistent storage mechanism. The Shuttle.ESB.SqlServer package contains an implementation of the ISubscriptionManager interface and this is what you are using. The publisher(s) and subscriber(s) can therefore also be grouped based on the subscription store but typically there would only be one.

The default connection name for the SqlServer subscription store is Subscription. The sqlServer configuration section allows you to change this for cases where you have a connection string used by your application that may have a different name. That is why you do not see the sqlServer section in the subscriber samples but only in the client where it is names SubscriptionConnection (the Connection being added). I probably could use something a bit different to make it somewhat more apparent.

It is not a bad idea to have a bit of a quick-start guide. I'll add an issue.

Please let me know if you have anymore questions.

[mpaul31]

thanks again eben that makes perfect sense.

is there any chance you can add the windsor domain handler implementation to the shuttle caste nuget?

On Fri, May 1, 2015 at 8:53 AM, Eben notifications@github.com wrote:

All subscriptions have to be stored in some persistent storage mechanism. The Shuttle.ESB.SqlServer package contains an implementation of the ISubscriptionManager interface and this is what you are using. The publisher(s) and subscriber(s) can therefore also be grouped based on the subscription store but typically there would only be one.

The default connection name for the SqlServer subscription store is Subscription. The sqlServer configuration section allows you to change this for cases where you have a connection string used by your application that may have a different name. That is why you do not see the sqlServer section in the subscriber samples but only in the client where it is names SubscriptionConnection (the Connection being added). I probably could use something a bit different to make it somewhat more apparent.

It is not a bad idea to have a bit of a quick-start guide. I'll add an issue.

Please let me know if you have anymore questions.

— Reply to this email directly or view it on GitHub https://github.com/Shuttle/shuttle-esb/issues/45#issuecomment-98127494.

[eben-roux]

Which Windsor domain handler are you referring to?

The IMessageHandlerFactory is implemented for shuttle here: https://www.nuget.org/packages/shuttle-esb-castle/

[mpaul31]

this guy

https://github.com/Shuttle/shuttle-core-domain-castle

On Fri, May 1, 2015 at 12:41 PM, Eben notifications@github.com wrote:

Which Windsor domain handler are you referring to?

The 'IMessageHandlerFactory' is implemented for shuttle here: https://www.nuget.org/packages/shuttle-esb-castle/

— Reply to this email directly or view it on GitHub https://github.com/Shuttle/shuttle-esb/issues/45#issuecomment-98174668.

[eben-roux]

Oops!

That chap shouldn't really exist anymore :)

I have come to the conclusion that the domain event dispatcher is not all that useful. I think a better technique is to return the event(s) from the domain methods.

See my SO answer on this:

http://stackoverflow.com/questions/28969371/domain-events-without-service-location/28978595#28978595

Let me know what you think.

[mpaul31]

lol ok cool!

so is your impementation of the IDomainEventDispatcher simply wrapping the bus and looping through the returned events and calling publish?

On Fri, May 1, 2015 at 12:47 PM, Eben notifications@github.com wrote:

Oops!

That chap shouldn't really exist anymore :)

I have come to the conclusion that the domain event dispatcher is somewhat of an anti-pattern. I think a better technique is to return the event(s) from the domain methods.

See my SO answer on this:

http://stackoverflow.com/questions/28969371/domain-events-without-service-location/28978595#28978595

— Reply to this email directly or view it on GitHub https://github.com/Shuttle/shuttle-esb/issues/45#issuecomment-98175597.

[eben-roux]

Well, I used to do all kinds of funny things in the handlers. But if the event is returned from the domain the application layer can do whatever is required:

• persistence
• service bus integration
• etc. (means there could be more but I don't know what it is)

Anyhow, that is why I don't know if dispatching to other handlers would add any value. I may, of course, change my mind should I come across a use case where it is useful but I think for now that within a single BC all the required work can be completed in the relevant application layer and publishing the event will result in any other interested BC getting the event and doing whatever voodoo it needs to do.

[mpaul31]

ok that makes sense.

i'm really digging how easy this is too get up and running. i remember trying this along time ago with NServiceBus and I think i broke a few things laying around my desk :)

something else I am trying to figure out here. I need to import a file based on a bunch of business rules and integrating with other BC's to get back relevant information which screams to me to be a long running process, but i know this is not built in too shuttle as this point.

so one option for me is going to be simply have a handler loop through all the records in the file and do what it needs to do and move on. So, this could be long running and I am trying to figure out all the timeout settings I need set for the sql command, sql connection, transaction scope (if shuttle uses this). i saw a timeout in the configuration section but not sure if this covers all 3 scenarios.

if you could point me in the right direction i would greatly appreciate it!

On Fri, May 1, 2015 at 1:39 PM, Eben notifications@github.com wrote:

Well, I used to do all kinds of funny things in the handlers. But if the event is returned from the domain the application layer can do whatever is required:

• persistence
• service bus integration
• etc. (means there could be more but I don't know what it is)

Anyhow, that is why I don't know if dispatching to other handlers would add any value. I may, of course, change my mind should I come across a use case where it is useful but I think for now that within a single BC all the required work can be completed in the relevant application layer and publishing the event will result in any other interested BC getting the event and doing whatever voodoo it needs to do.

— Reply to this email directly or view it on GitHub https://github.com/Shuttle/shuttle-esb/issues/45#issuecomment-98184758.

[eben-roux]

Thanks for the thumbs up :)

With most long running processes you are looking at multiple messages to complete a single use case. That use case requires state to keep track of what is going on. There are various ways to get this done. One is to use a process manager (aka a Saga --- although that originated in the NServiceBus world) --- this is orchestration. Another is to use a "routing slip" idea where you keep the state in the message --- this is closer to choreography.

I know other service buses have this "built-in". NServiceBus and MassTransit uses a saga but it is implemented slightly differently. I have been struggling to see too much use in this but I have always maintained that using event sourcing would be the most appropriate "generic" solution. This is why I started with my shuttle-recall project since I am experimenting with a different take on the "traditional" ES mechanism.

Anyway. Any generic process handling will implemented using modules so that one could opt for whatever mechanism you are comfortable with. I don't want anything in the core as this will bloat it with a particular opinionated mechanism.

That being said: when I have needed a process manager I have simply implemented it myself.

So when a process "start" message is handled I created an instance of my process aggregate as a first-class citizen and persisted it using the repository. Each message that is involved in the process would simply load the process aggregate and perform the requisite behaviour and then persist it.

For what NServiceBus calls "Timeouts" you can use the normal deferred messages for later processing. If you do it is best to use a deferred queue.

The timeout in the configuration section is for the transaction scope. I'll have to take a look at where the rest can be defined and get back to you. Please remind me if I forget :)

So I still do not know whether I'll have a particular module for the "sagas". I have a friend that has actually implemented sagas using attributes and reflection emit but he hasn't made it public. Perhaps I can prod him along.

Hope that helps.

[eben-roux]

For my own database interaction code I make use of the Shuttle.Core.Data package and the only other timeout it sets is for the command timeout that is defaulted to 15 but can be set using an appSetting key since it is read as follows:

private static readonly int commandTimeout = ConfigurationItem<int>.ReadSetting("Shuttle.Core.Data.DbCommandFactory.CommandTimeout", 15).GetValue();

[mpaul31]

Awesome this should help me get going. So do u just use the default timeout for the connection ?

Sent from my iPhone

On May 2, 2015, at 4:51 AM, Eben notifications@github.com wrote:

For my own database interaction code I make use of the Shuttle.Core.Data package and the only other timeout it sets is for the command timeout that is defaulted to 15 but can be set using an appSetting key since it is read as follows:

private static readonly int commandTimeout = ConfigurationItem.ReadSetting("Shuttle.Core.Data.DbCommandFactory.CommandTimeout", 15).GetValue(); — Reply to this email directly or view it on GitHub.

[mpaul31]

So let's say u need a long running process for an invoice. Do u have two objects one for the domain object and another for the state?

[eben-roux]

I haven't looked at the database code for quite some time but I do not specify the connection timeout. It could be that I reckoned, at the time, that the default should be OK.

If you have a process for an Invoice I would have the process state separate, yes. It could even be that you are interacting with other BCs so the process "layer" could be a BC of its own since it is interacting with multiple BCs.

Domain objects have interesting interactions, though. For an "Order" process you could go from a ShoppingCart or a Quote to an Order to an Invoice where all the objects are part of the same business process but one leads into the next.

In some instances you may not even have enough data for the "actual" domain object and you need to gather all the bits and perhaps perform various bits of processing before you can register/create the "actual" domain object. The process state would be where you keep that data until it has all been gathered.

[mpaul31]

eben,

it looks like my journal queues are always empty, are they not suppose to keep a copy of all sent messages? When I look at the queue from the computer management the enabled checkbox for journal is not checked.

thanks

On Sat, May 2, 2015 at 12:48 PM, Eben notifications@github.com wrote:

I haven't looked at the database code for quite some time but I do not specify the connection timeout. It could be that I reckoned, at the time, that the default should be OK.

If you have a process for an Invoice I would have the process state separate, yes. It could even be that you are interacting with other BCs so the process "layer" could be a BC of its own since it is interacting with multiple BCs.

Domain objects have interesting interactions, though. For an "Order" process you could go from a ShoppingCart or a Quote to an Order to an Invoice where all the objects are part of the same business process but one leads into the next.

In some instances you may not even have enough data for the "actual" domain object and you need to gather all the bits and perhaps perform various bits of processing before you can register/create the "actual" domain object. The process state would be where you keep that data until it has all been gathered.

— Reply to this email directly or view it on GitHub https://github.com/Shuttle/shuttle-esb/issues/45#issuecomment-98377507.

[eben-roux]

Hi,

It depends which journal queues you are referring to. Shuttle doesnot make use of MSMQ journal queues but the default MSMQQueue does create $journal queues for the at-least-once delivery mechanism.

Those would typically be empty since they are used to ensure that messages are not lost while being handled. So only while a message is handled is it in the $journal queue and then removed once acknowledged; else it is moved back to the source queue from the journal queue when it is somehow interrupted (power down etc.) --- so messages between source and journal use a MSMQ transaction which is a rather quick affair.

You haven't by any chance considered RabbitMQ --- quite simple to get going and very fast :)

[mpaul31]

ok thanks that makes sense i just thought i read somewhere in the shuttle docs to be sure to create a journal queue if you plan on creating your queues upfront (production).

no i have not. the only reason is because we are a microsoft shop but this is something i can definitely look into.

also, one more thing. if i have an application service that is directly called from the UI and let's say i add something to a repository and then publish an event on the bus. obviously this is not atomic, so i was wondering if it is a best practice to always do multi-step things like this from within an message handler. i think otherwise i would need to use DTC but i remember you saying you removed this from shuttle.

On Mon, May 4, 2015 at 8:39 AM, Eben notifications@github.com wrote:

Hi,

It depends which journal queues you are referring to. Shuttle doesnot make use of MSMQ journal queues but the default MSMQQueue does create $journal queues for the at-least-once delivery mechanism.

Those would typically be empty since they are used to ensure that messages are not lost while being handled. So only while a message is handled is it in the $journal queue and then removed once acknowledged; else it is moved back to the source queue from the journal queue when it is somehow interrupted (power down etc.) --- so messages between source and journal use a MSMQ transaction which is a rather quick affair.

You haven't by any chance considered RabbitMQ --- quite simple to get going and very fast :)

— Reply to this email directly or view it on GitHub https://github.com/Shuttle/shuttle-esb/issues/45#issuecomment-98696204.

[eben-roux]

Ah, yes! That is a tricky business.

The simplest is to have a single "thing" happen. So sending a message and then having the application server do the rest of the processing is always going to be the easiest.

I have given it some thought and couldn't really come up with any solution that resembles anything atomic where one performs any operation and then also send a message.

So I think the single message is going to be the best.

So instead of:

• user call (say rest post)
• insert record
• send message
• hope both happened
• tell user request accepted (e.g. respond 200 - OK)

We could have:

• user call (say rest post)
• send message
• tell user request accepted (e.g. respond 200 - OK)
• message will process so insert record (asynchronous - eventually consistent)

[mpaul31]

cool that's what i thought. little off topic, but for your domain persistence do you use entity framework or a home grown soltuion? man i want to use EF but i am running into so much friction!

On Mon, May 4, 2015 at 9:02 AM, Eben notifications@github.com wrote:

Ah, yes! That is a tricky business.

The simplest is to have a single "thing" happen. So sending a message and then having the application server do the rest of the processing is always going to be the easiest.

I have given it some thought and couldn't really come up with any solution that resembles anything atomic where one performs any operation and then also send a message.

So I think the single message is going to be the best.

So instead of:

• user call (say rest post)
• insert record
• send message
• hope both happened
• tell user request accepted (e.g. respond 200 - OK)

We could have:

• user call (say rest post)
• send message
• tell user request accepted (e.g. respond 200 - OK)
• message will process so insert record (asynchronous - eventually consistent)

— Reply to this email directly or view it on GitHub https://github.com/Shuttle/shuttle-esb/issues/45#issuecomment-98699890.

[eben-roux]

For my persistence I stay away from any ORM. I use my shuttle-core-data package. Super simple and lightweight. A slight abstraction over ADO.NET and uses the provider factories, etc.

[mpaul31]

awesome i'll take a look

On Mon, May 4, 2015 at 9:34 AM, Eben notifications@github.com wrote:

For my persistence I stay away from any ORM. I use my shuttle-core-data package. Super simple and lightweight. A slight abstraction over ADO.NET and uses the provider factories, etc.

— Reply to this email directly or view it on GitHub https://github.com/Shuttle/shuttle-esb/issues/45#issuecomment-98705486.

[mpaul31]

looks really cool. i'm assuming not, but is there any sample code out there showing how this is used with your aggregates and value objects?

On Mon, May 4, 2015 at 9:50 AM, Marco Paul mpaul31@gmail.com wrote:

awesome i'll take a look

On Mon, May 4, 2015 at 9:34 AM, Eben notifications@github.com wrote:

For my persistence I stay away from any ORM. I use my shuttle-core-data package. Super simple and lightweight. A slight abstraction over ADO.NET and uses the provider factories, etc.

— Reply to this email directly or view it on GitHub https://github.com/Shuttle/shuttle-esb/issues/45#issuecomment-98705486.

[eben-roux]

No real samples although I have used it for all my Sql Server implementations and there are some unit tests. I really should improve the docs but, then again, it has never been something that I am pushing for general use; more something I used for myself although anyone is more than welcome to use it :)

[mpaul31]

ok cool i'll take a look at those packages.

On Mon, May 4, 2015 at 10:45 AM, Eben notifications@github.com wrote:

No real samples although I have used it for all my Sql Server implementations and there are some unit tests. I really should improve the docs but, then again, it has never been something that I am pushing for general use; more something I used for myself although anyone is more than welcome to use it :)

— Reply to this email directly or view it on GitHub https://github.com/Shuttle/shuttle-esb/issues/45#issuecomment-98736728.