ESB duplicate messages

[andyhracti]

I'm currently using ESB V10.0.10, Sql queue, 10.1.1, SQL.Subscriuption 10.0.0.1. to handle a 'Published' message. Dependancy injection is using Autofac.

When message processing fails and is deferred, we get duplicate messages being created. This is made worse with smaller values of 'durationToIgnoreOnFailure', and is inconsistent (the number of duplicate messages is not the same). Also the maximum failure count doesn't seem to be applied when this occurs. The issue does not occur when debugging in visual studio, but does occur on deployment as a windows service or running as a console.

We can stop this by setting threadCount="1" on the 'inbox' config settings, but we feel it should be able to run under the default behavior .

This looks to be a bug unless we are missing some essential requirements/techniques for running the default threadcount.

[eben-roux]

My last change to the SqlQueue was to allow it to function in a broker-style. I may have introduced a bug at that point so I'll definitely have a look as soon as I can.

Just some questions:

• Do you feed off the queue using multiple endpoints or only from a single endpoint?
• If you are using multiple endpoints do they each have their own deferred queue?

I'm assuming it published fine (one message) and that your duplicates are on the subscriber.

[andyhracti]

Hi Eben,

I can confirm I’ve only one endpoint. (shut down all apps and checked for other connection with SQL profiler)

We code our services to have separate work/defer/error queues.

Ruled out the sending application with a test app…. Definitely sending one message via “Publish”

It is possibly easier to replicate , on slower/busier machines or when restarting the subscriber process/service.

Thanks

[eben-roux]

Right, so it turns out that there was a bug :)

A select followed by an update is usually not the best idea.

In any event, please give the latest versions of the Shuttle.Esb, Shuttle.Esb.Sql.Queue and Shuttle.Esb.Sql.Subscription packages a go in a test environment (I guess that goes withut saying).

I'm not too thrilled by the performance but for now getting rid of the duplicates should be the priority.

[andyhracti]

Thanks Eben!

That seems to be working much better now… I’ve not managed to replicate the issue on the latest libraries.

Regards Andy

[eben-roux]

Fixed in https://github.com/Shuttle/Shuttle.Esb.Sql.Queue/releases/tag/v10.1.2

Please re-open or log a new issue if you still find this to be an issue.