CreateConnection() seemingly fails to connect when run in an async context with Nito.AsyncEx

[JohannesRudolph]

I'm using the Nito.AsyncEx library (https://github.com/StephenCleary/AsyncEx) to provide an Async Context for my workers (.netcore console app).

See this thread for more info on why this is necessary and a good idea: http://stackoverflow.com/questions/9208921/cant-specify-the-async-modifier-on-the-main-method-of-a-console-app/9212343#9212343

In any case, this throws a BrokerUnreachableException

        static void Main( string[] args )
        { 
            AsyncContext.Run( () =>var mq = new RabbitMqConfiguration( isTest: false ).ConnectionFactory.CreateConnection(); );
        }

while this does not:

        static void Main( string[] args )
        {  
             var mq = new RabbitMqConfiguration( isTest: false ).ConnectionFactory.CreateConnection();
        }

Probably a bug in RabbitMqClient. I've tried versions 4.0, 4.1 at latest rc1 at the time of writing.

[michaelklishin]

Please post questions to rabbitmq-users or Stack Overflow. RabbitMQ uses GitHub issues for specific actionable items engineers can work on, not questions. Thank you.

[michaelklishin]

There is no evidence that this is a RabbitMQ client bug because it only can be reproduced when a particular external library is used. Please post a runnable code example to the mailing list and maybe someone will be able to figure out what's going on.

[JohannesRudolph]

Your call. This bug report contains all facts needed to reproduce. Replace RabbitMqConfiguration(...).ConnectionFactory with

new ConnectionFactory()
            {
                HostName = "192.168.99.100",
                Port = AmqpTcpEndpoint.UseDefaultPort,
                Protocol = Protocols.DefaultProtocol,
                RequestedConnectionTimeout = 2000, // fail the test early
                VirtualHost = "/",
                UserName = "user",
                Password ="password"
}

or whatever is appropriate in your dev environment. Nito.AsyncEx is a very popular package to write proper async context enabled console apps.

Maybe rabbitmq forgets to call .ConfigureAwait(false) on some internal async code? This is a common problem for libraries that can't assume too much about the async context they run in. See also https://msdn.microsoft.com/en-us/magazine/jj991977.aspx

Feel free to investigate or leave it at that, my workaround for now is to create the connection outside of the async context.

[michaelklishin]

@JohannesRudolph sorry but you may be overestimating the usefulness of a single code snippet and a link to a library.

We did add a few missing ConfigureAwaits before in https://github.com/rabbitmq/rabbitmq-dotnet-client/issues/239 and haven't heard this reported since. So I somewhat doubt it is an obvious issue. Regardless of how popular Nito.AsyncEx is (I cannot really evaluate it), having a runnable example would help immensely.

[JohannesRudolph]

Here's a full repro project: ConsoleApp1.zip

[kjnilsson]

@JohannesRudolph thanks for the repro project. I tested it and after changing the HostName and UserName/Password parameters it connects fine for me when doing dotnet run.

Output:

Connected Sync
Connected async

As @michaelklishin mentioned, we added some more ConfigureAwait(false)s recently but even so a console app shouldn't be affected by this because of the SynchronizationContext it uses. I haven't looked into how Nitro dispatches the action provided but can't see how it would affect the connection phase. Not sure what else to suggest. Are you running this on windows or a unix variant?

[michaelklishin]

If this is a race condition we should try running it 1000 times or so ;)

[JohannesRudolph]

I'm running on Windows, get the exception everytime:

PS C:\Users\johannes.rudolph\Documents\Visual Studio 2015\Projects\ConsoleApp1\src\ConsoleApp1> dotnet run
Project ConsoleApp1 (.NETCoreApp,Version=v1.0) will be compiled because expected outputs are missing
Compiling ConsoleApp1 for .NETCoreApp,Version=v1.0

Compilation succeeded.
    0 Warning(s)
    0 Error(s)

Time elapsed 00:00:01.6215815

Connected Sync

Unhandled Exception: RabbitMQ.Client.Exceptions.BrokerUnreachableException: None of the specified endpoints were reachab
le ---> RabbitMQ.Client.Exceptions.ConnectFailureException: Connection failed ---> System.TimeoutException: The operatio
n has timed out.
   at RabbitMQ.Client.Impl.TaskExtensions.<TimeoutAfter>d__0.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at RabbitMQ.Client.Impl.SocketFrameHandler.Connect(ITcpClient socket, AmqpTcpEndpoint endpoint, Int32 timeout)
   --- End of inner exception stack trace ---
   at RabbitMQ.Client.EndpointResolverExtensions.SelectOne[T](IEndpointResolver resolver, Func`2 selector)
   at RabbitMQ.Client.ConnectionFactory.CreateConnection(IEndpointResolver endpointResolver, String clientProvidedName)
   --- End of inner exception stack trace ---
   at RabbitMQ.Client.ConnectionFactory.CreateConnection(IEndpointResolver endpointResolver, String clientProvidedName)
   at ConsoleApp1.Program.<>c__DisplayClass0_0.<Main>b__0()
   at System.Threading.Tasks.Task.Execute()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Nito.AsyncEx.AsyncContext.Run(Action action)
   at ConsoleApp1.Program.Main(String[] args)
PS C:\Users\johannes.rudolph\Documents\Visual Studio 2015\Projects\ConsoleApp1\src\ConsoleApp1>

I haven't heard that console apps had a SynchronizationContext, but this may be different in dotnet core....

[kjnilsson]

If the SynchronisationContext isn't set then it uses the default context (threadpool dispatch). See: https://msdn.microsoft.com/magazine/gg598924.aspx

Tried it 1000 times and it still succeeds. Will try to do some more digging inside a windows VM.

[kjnilsson]

Works for me on windows. @JohannesRudolph can you test this against a localhost instance of rabbit? Does it fail there as well?

[michaelklishin]

If someone has a way to reproduce it, please leave a comment.