Groups.Add(...) never completes

[baralong]

In our solution we have clients using groups like topics to subscribe to items of interest.

Unfortunately when I added the RabbitMq backplane it stopped working. This gist https://gist.github.com/baralong/6822725 has replacements for Index.cshtml and chat.cs that reproduce the problem. comment out GlobalHost.DependencyResolver.UseRabbitMq(configuration); in Global.asax.cs then it all works.

Tracing through it looks like the problem is that SignalR requires an Ack for group join messages. In Microsoft.AspNet.SignalR.Infrastructure.Connection.Send(...) it waits for the task from the sending of the message on the bus to complete. But the TaskCompletionSource

[mdevilliers]

Thanks for the gist to replicate. I'll have a look asap.

[mdevilliers]

Hey,

I've had a go at replicating your bug and checked in your jist changes to a new branch issue13 at https://github.com/mdevilliers/SignalR.RabbitMq/tree/issue13

Unfortunately the example runs correctly for me -

Could you double check I've set this up correctly?

I ran my test using the following versions -

RabbitMQ 3.1.5, Erlang R16B02 SignalR 1.1.3 SignalR.RabbitMQ 2.1.3

In the RabbitMQ management UI I can see the rabbitmq-stamp plugin is installed.

Thanks,

Mark

[baralong]

Hi Mark,

Thanks for the quick response. Unfortunately its on my work machine and I'm not set up for it here at home (it Friday evening for me here) I'll try and get the environment set up on a different machine in the morning.

Cheers, Doug On Oct 4, 2013 8:35 PM, "Mark deVilliers" notifications@github.com wrote:

Hey,

I've had a go at replicating your bug and checked in your jist changes to a new branch issue13 at https://github.com/mdevilliers/SignalR.RabbitMq/tree/issue13

Unfortunately the example runs correctly for me -

[image: issue13]https://f.cloud.github.com/assets/98798/1269052/8beb0554-2cf0-11e3-9bd4-c980e7eeea33.png

Could you double check I've set this up correctly?

I ran my test using the following versions -

RabbitMQ 3.1.5, Erlang R16B02 SignalR 1.1.3 SignalR.RabbitMQ 2.1.3

In the RabbitMQ management UI I can see the rabbitmq-stamp plugin is installed.

[image: image]https://f.cloud.github.com/assets/98798/1269060/bcffb8d8-2cf0-11e3-9cb2-03a8f460a428.png

Thanks,

Mark

— Reply to this email directly or view it on GitHubhttps://github.com/mdevilliers/SignalR.RabbitMq/issues/13#issuecomment-25695273 .

[baralong]

Hi Mark,

I've set things up on an different PC and it's all working fine. I'll have to check the setup on my work PC on Monday.

I'll let you know how I go.

Thanks, Doug

On 4 October 2013 20:51, Doug Paice doug.paice@baralong.org wrote:

Hi Mark,

Thanks for the quick response. Unfortunately its on my work machine and I'm not set up for it here at home (it Friday evening for me here) I'll try and get the environment set up on a different machine in the morning.

Cheers, Doug On Oct 4, 2013 8:35 PM, "Mark deVilliers" notifications@github.com wrote:

Hey,

I've had a go at replicating your bug and checked in your jist changes to a new branch issue13 at https://github.com/mdevilliers/SignalR.RabbitMq/tree/issue13

Unfortunately the example runs correctly for me -

[image: issue13]https://f.cloud.github.com/assets/98798/1269052/8beb0554-2cf0-11e3-9bd4-c980e7eeea33.png

Could you double check I've set this up correctly?

I ran my test using the following versions -

RabbitMQ 3.1.5, Erlang R16B02 SignalR 1.1.3 SignalR.RabbitMQ 2.1.3

In the RabbitMQ management UI I can see the rabbitmq-stamp plugin is installed.

[image: image]https://f.cloud.github.com/assets/98798/1269060/bcffb8d8-2cf0-11e3-9cb2-03a8f460a428.png

Thanks,

Mark

— Reply to this email directly or view it on GitHubhttps://github.com/mdevilliers/SignalR.RabbitMq/issues/13#issuecomment-25695273 .

[mdevilliers]

Cool. That's good news. Tell me how you get on. On 4 Oct 2013 23:35, "Doug Paice" notifications@github.com wrote:

Hi Mark,

I've set things up on an different PC and it's all working fine. I'll have to check the setup on my work PC on Monday.

I'll let you know how I go.

Thanks, Doug

On 4 October 2013 20:51, Doug Paice doug.paice@baralong.org wrote:

Hi Mark,

Thanks for the quick response. Unfortunately its on my work machine and I'm not set up for it here at home (it Friday evening for me here) I'll try and get the environment set up on a different machine in the morning.

Cheers, Doug On Oct 4, 2013 8:35 PM, "Mark deVilliers" notifications@github.com wrote:

Hey,

I've had a go at replicating your bug and checked in your jist changes to a new branch issue13 at https://github.com/mdevilliers/SignalR.RabbitMq/tree/issue13

Unfortunately the example runs correctly for me -

[image: issue13]< https://f.cloud.github.com/assets/98798/1269052/8beb0554-2cf0-11e3-9bd4-c980e7eeea33.png>

Could you double check I've set this up correctly?

I ran my test using the following versions -

RabbitMQ 3.1.5, Erlang R16B02 SignalR 1.1.3 SignalR.RabbitMQ 2.1.3

In the RabbitMQ management UI I can see the rabbitmq-stamp plugin is installed.

[image: image]< https://f.cloud.github.com/assets/98798/1269060/bcffb8d8-2cf0-11e3-9cb2-03a8f460a428.png>

Thanks,

Mark

— Reply to this email directly or view it on GitHub< https://github.com/mdevilliers/SignalR.RabbitMq/issues/13#issuecomment-25695273>

.

— Reply to this email directly or view it on GitHubhttps://github.com/mdevilliers/SignalR.RabbitMq/issues/13#issuecomment-25734702 .

[baralong]

I haven't checked things out on my work PC, but I did notice one thing: I never hit a breakpoint in either the "done" or "fail" method on the "joinGroup" call in the JavaScript, also I sometimes get an issue where my browser window hangs when I try and close it after clicking "joinGroup", but that last bit may be unrelated.

I tried adding "tcs.SetResult(null);" before the return in Send in RabbitMqMessageBus and it seems to fix both problems, but that's a bit of a hack. What I think a better solution would be is to pass the task along with the RabbitMqMessageBuffer so that it can be completed once it's sent in the loop in "ConnectToRabbit" as that might be the right place to Ack that the message has been sent. Does that make sense?

I might see if I can dig into one of the other SignalR bus options to see what they do, but I haven't had a chance as yet, I'll try it tomorrow and let you know how it goes. On Oct 5, 2013 11:16 PM, "Mark deVilliers" notifications@github.com wrote:

Cool. That's good news. Tell me how you get on. On 4 Oct 2013 23:35, "Doug Paice" notifications@github.com wrote:

Hi Mark,

I've set things up on an different PC and it's all working fine. I'll have to check the setup on my work PC on Monday.

I'll let you know how I go.

Thanks, Doug

On 4 October 2013 20:51, Doug Paice doug.paice@baralong.org wrote:

Hi Mark,

Thanks for the quick response. Unfortunately its on my work machine and I'm not set up for it here at home (it Friday evening for me here) I'll try and get the environment set up on a different machine in the morning.

Cheers, Doug On Oct 4, 2013 8:35 PM, "Mark deVilliers" notifications@github.com wrote:

Hey,

I've had a go at replicating your bug and checked in your jist changes to a new branch issue13 at https://github.com/mdevilliers/SignalR.RabbitMq/tree/issue13

Unfortunately the example runs correctly for me -

[image: issue13]<

https://f.cloud.github.com/assets/98798/1269052/8beb0554-2cf0-11e3-9bd4-c980e7eeea33.png>

Could you double check I've set this up correctly?

I ran my test using the following versions -

RabbitMQ 3.1.5, Erlang R16B02 SignalR 1.1.3 SignalR.RabbitMQ 2.1.3

In the RabbitMQ management UI I can see the rabbitmq-stamp plugin is installed.

[image: image]<

https://f.cloud.github.com/assets/98798/1269060/bcffb8d8-2cf0-11e3-9cb2-03a8f460a428.png>

Thanks,

Mark

— Reply to this email directly or view it on GitHub<

https://github.com/mdevilliers/SignalR.RabbitMq/issues/13#issuecomment-25695273>

.

— Reply to this email directly or view it on GitHub< https://github.com/mdevilliers/SignalR.RabbitMq/issues/13#issuecomment-25734702>

.

— Reply to this email directly or view it on GitHubhttps://github.com/mdevilliers/SignalR.RabbitMq/issues/13#issuecomment-25750374 .

[baralong]

I've found that I still have the problem when I connect to our staging environment. I've implemented a few changes to pass the TaskCompletionSource in the message wrapper and then completing it or passing an exception in the. This helped with getting a response from JoinGroup, but that's about it.

Debugging the example in my staging environment I can see the messages being sent to the queue but I never get anything back. I'm starting to think it's something to do with the way our cluster is set up. It's a bit odd, I've had no trouble with MassTransit sending messages.

Perhaps it's something in the routing, I'm afraid I'm I little out of my depth with Rabbit at this point.

[baralong]

Sorry this has gotten a bit off topic.. I think I've found my problem, I don't then the stamp exchange is forwarding the message onto the regular one. I can see the stamp-x getting the message, but nothing appears anywhere else.

Is there anything in the management console I can try to test it out? I do see the signalr-stamp exchange with a type of x-stamp a policy of "ha-all" and parameters of "D"

Thanks

[mdevilliers]

You can send a message to the signalr-stamp queue

• click on Exchanges and create an exchange
• click on queues and create a debugging queue

• select the exchange from step 1, attach the debugging queue to the exchange you created in step 1

• select the signalr-stamp queue and send a message with the appropriate header

• go back to your logging queue and click on Get Messages and the message should be there

I'm interested in your policy of "ha-all" not being an expert on the ha side of rabbit. Maybe that is the key?

[mdevilliers]

Another thing might be to look in the rabbit logs

on linux /var/log/rabbitmq on windows something like C:\Users{user}\AppData\Roaming\RabbitMQ\log

Mark

[baralong]

I think I've had some success, going through your steps on my local machine I get the message ending up in the log queue.

After that I created a virtual host (called "vhost") and added the similarly named 2 exchanges and queue. When I published to the vhost/signalr-stamp the message ended up in the log that wasn't part of the vhost.

What I suspect is that the message published to the x-stamp exchange needs to have the virtual host included in the name of the exchange or as another header, or the exchange could pick it up from its own virtual host.

I'm not familiar with erlang, but looking at https://github.com/mdevilliers/rabbitmq-stamp/blob/master/src/rabbit_stamp_worker.erl line 51

 {_Ok, Msg} = rabbit_basic:message({resource,<<"/">>, exchange, ToExchange}, RoutingKey, Content1),

It looks like the exchange may be hard coded.

Could you see if you can duplicate this issue? If you can, could the stamp plug-in be modified to take into account the virtual host?

thanks, Doug

[baralong]

BTW I did have a look at the logs on my machine and I didn't see anything related to the stamp worker, other than it staring.

[baralong]

I got our administrator to give the signalR user permissions to the default queue, so it looks like that's the solution. Should I create an issue over on rabbitmq-stamp?

[mdevilliers]

Thanks for all your work on this. I'll get the rabbitmq-stamp plugin fixed - great spot!

I'll keep this issue open until it is fixed (which hopefully will be today)

Thanks again,

Mark

[mdevilliers]

Can you try out the new version of the rabbitmq-stamp plugin?

It is at - https://github.com/mdevilliers/rabbitmq-stamp/tree/master/compiled/1.0.2/RabbitMQ_3_1

It now captures the virtual host the original message was sent to. One thing to look out for is to delete/remove the version 1.0.1 plugin as rabbit will load the first one it comes accross.

It would be great to get some feedback and if it works for you I'll update the nuget package?

Thanks for all your effort on this.

[baralong]

Works for me :)

Thanks for the quick turnaround on the solution. Glad I could dig into it, I've learnt quite a bit about RabbitMQ and SignalR in the process, well quite a bit more than I knew before at least.

[baralong]

I've just sent a pull request for the TaskCompletionSource code I have running in my local version. Let me know how it looks.

[mdevilliers]

I'll just close this as both your and my testing looks ok.