The problem after updating from rc0002 to release version

[Lapov]

Hi guys! I use SqlPersistance to store saga data to the MSSQL DB. After I made an update from rc0002 to release version. I made next actions:

1. Remove old saga tables
2. Update all packages.
3. Clean/rebuild twice =)
4. EndpointInstance does not launch. It just hangs in endpointStart.Wait(). Without any exceptions/logs.

My config:

var endpointConfiguration = new EndpointConfiguration("worker");
endpointConfiguration.EnableInstallers();
endpointConfiguration.UsePersistence<InMemoryPersistence>();

var persistence = endpointConfiguration.UsePersistence<SqlPersistence, StorageType.Sagas>();
persistence.TablePrefix(string.Empty);
persistence.SqlVariant(SqlVariant.MsSqlServer);
persistence.ConnectionBuilder(() => new SqlConnection("ConnectionString"));
persistence.DisableInstaller(); // <- first run without this line. Then add it.
...
var endpointStart = Endpoint.Start(endpointConfiguration);
endpointStart.Wait();  //<- it hangs at this line
return endpointStart.Result

I use NServiceBus.SqlPersistence.MsBuild to generate a Saga tables. And they actually created if I do not use persistence.DisableInstaller() in configuration.

After rolling back to the rc0002 it works fine. What am I doing wrong?

Thank you.

[SimonCropp]

@gromy

both wait and result are known to cause deadlocks.

are you hosting in a windows service? if so can u try this approach https://docs.particular.net/nservicebus/hosting/windows-service there is a downloadable sample here https://docs.particular.net/samples/hosting/windows-service/

i note u use UsePersistence<InMemoryPersistence>();. it is unusual to mix durable and non durable storages for different StorageTypes

generally DisableInstaller should be wrapped in some logic that identifies a production environment https://docs.particular.net/nservicebus/sql-persistence/installer-workflow#contrasting-workflows https://docs.particular.net/nservicebus/operations/installers

[Lapov]

@SimonCropp Thank you for fast response.

I use Topshelf to host it. I will try to change wait/result to ConfigureAwait(false).GetAwaiter().GetResult() and tell you know if it helps me.

Actually I need to store SagaData in MSSQL Db and other staff in RabbitMQ like Subscriptions for example. I thought it is ok to use it in this way...

Actually we will use DisableInstaller every time and will update Db manually to manage it completely.

[SimonCropp]

Actually I need to store SagaData in MSSQL Db and other staff in RabbitMQ like Subscriptions for example. I thought it is ok to use it in this way...

Yep that is fine. and now i understand why u have InMemoryPersistence in this snippet

[Lapov]

@SimonCropp About persistence: cool. About hosting: I checked .ConfigureAwait(false).GetAwaiter().GetResult(). The behavior completely the same. And actually it doesnot metter which hosting I will use GetResult() or wait/result should return constructed EndpointInstance. In service it will be run or in console app. the problem is that it is hangs while this construction =( And yeah - rc0002 works fine. Can you give me more information about about "wait and result are known to cause deadlocks", please.

[SimonCropp]

http://blog.stephencleary.com/2012/07/dont-block-on-async-code.html

You should be using await to get the endpoint instance.

[SimonCropp]

Can u upload a small repro to GitHub including top shelf and I will debug it

[Lapov]

Hi @SimonCropp I made some research and have found that the problem related with conjunction SQLPersistance and Autofac named child life time scope. If I remove InstancePerMatchingLifetimeScope(scopeName) and scopeName form BeginLifetimeScope (use just child lifetime scope without name) it works fine. I made an example as simple as possible:

var builder = new ContainerBuilder();
var container = builder.Build();
var scopeName = "newLife";
var newScope = container.BeginLifetimeScope(scopeName, child =>
{
    child.Register(scope =>
    {
        var endpointConfiguration = new EndpointConfiguration("worker");
        endpointConfiguration.EnableInstallers();
        endpointConfiguration.SendFailedMessagesTo("worker.error");
        endpointConfiguration.UsePersistence<InMemoryPersistence>();

        var persistence = endpointConfiguration.UsePersistence<SqlPersistence, StorageType.Sagas>();
        persistence.SqlVariant(SqlVariant.MsSqlServer);
        persistence.ConnectionBuilder(() => new SqlConnection(@"connectionString"));

        endpointConfiguration.UseTransport<RabbitMQTransport>().ConnectionString("connectionString");

        endpointConfiguration.UseContainer<AutofacBuilder>(
            customizations: customizations =>
            {
                customizations.ExistingLifetimeScope(scope.Resolve<ILifetimeScope>());
            });

        var endpoint = Endpoint.Start(endpointConfiguration).ConfigureAwait(false).GetAwaiter().GetResult();
        return endpoint;
    }).As<IEndpointInstance>()
     .InstancePerMatchingLifetimeScope(scopeName)
     .OnRelease(instance => instance.Stop().Wait());
});
newScope.Resolve<IEndpointInstance>();
return true;

I just wanna easy release all objects created by EndpointInstance just disposing child lifetime scope.

[SimonCropp]

@gromy but IEndpointInstance (and all its child parts) should be scoped once per appdomain. ie singleton scope

[Lapov]

@SimonCropp Can you give me more information about why I should use it once per AppDomain please? Enable hosting multiple nsb-runtimes in the same appdomain "This is now possible as part of v5" It means I at least I can use multiple instance in one process, it is not? I use version 6. And there there is an example of using two IEndpointInstances in one application

And it works fine with in memory persistence and with Sql persistence too. But when I wanna use both types of endpoints (for integration tests for example), the endpoint with only in memory persistence fails to run (but it is already another issue =) ).

[SimonCropp]

yes multiple different endpoints in the same appdomain are supported. but you are spinning up multiple instances of the same endpoint EndpointConfiguration("worker");

with multiple instances of the same endpoint you are effectively creating a competing consumer scenario but with none of the benefits since they are sharing the same hardware.

[Lapov]

@SimonCropp actually you right. But if we want to change the configuration of endpoint in run time, we just can use one endpoint, create another one with the different configuration and then stop the first. And some time they can work simultaneously. Just one scenario why we need to use it. Any way, now I have a problem that if I try to launch two different endpoint (with different names) in one application and want to setup one of it only for sending messages with in memory persistence and another with Saga persistence I got a problem with launch first one. It throw an exception ConnectionBuilder must be defined. With next call stack:

   at NServiceBus.SqlPersistenceConfig.GetConnectionBuilder(ReadOnlySettings settings) in C:\projects\nservicebus-sqlpersistence\SqlPersistence\Config\SqlPersistenceConfig_Connection.cs:line 25
   at Installer.<Install>d__3.MoveNext() in C:\projects\nservicebus-sqlpersistence\SqlPersistence\Installer.cs:line 24
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at NServiceBus.InitializableEndpoint.<RunInstallers>d__13.MoveNext() in C:\Build\src\NServiceBus.Core\InitializableEndpoint.cs:line 164
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at NServiceBus.InitializableEndpoint.<Initialize>d__1.MoveNext() in C:\Build\src\NServiceBus.Core\InitializableEndpoint.cs:line 58
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at NServiceBus.Endpoint.<Start>d__1.MoveNext() in C:\Build\src\NServiceBus.Core\Endpoint.cs:line 27

It is another topic, should I create different issue for it?

And just remind that the actual question was related with using InstancePerMatchingLifetimeScope(scopeName). Pure NServiceBus allows us to use it. rc2 version of SQLPersistence worked fine with named scopes too. But release version does not work fine.

Another interesting information: when I use rc2 version and remove persistence.DisableInstaller(); the same situation is exist: the tables are created, but the Endpoint does not start.(hangs in Endpoint.Start())

[SimonCropp]

@Lapov i would still like to get to the root cause of you issue. but am having trouble reproducing it. would you be able to share a stand alone repro that causes the hang

[Lapov]

@SimonCropp ok, check it, please =) https://github.com/Lapov/SqlPersistenceTest

[SzymonPobiega]

@Lapov what is the actual thing you want to achieve here with this code? Do you want multiple endpoints in the same host? Most of the times, especially when hosting outside of a web app, you don't actually need to put the whole endpoint instance in a container.

I've run the code you provided and it hangs. I made it pass by creating yet-another-lifetime-scope and passing it to NSB but I don't think this is the way to go. I am pretty sure the fact it hangs is related to the fact that the lifetime scope is being access when resolving the instance.

[Lapov]

Ok, thank you guys.