Suddenly breaking, though nothing has changed

[sbmw]

Hi, I have a site that's been running fine for years, but suddenly if services are restarted, Rebus will throw exceptions such as:

Application startup exception

Rebus.Injection.ResolutionException: Could not resolve Rebus.Transport.ITransport with decorator depth 0 - registrations: Rebus.Injection.Injectionist+Handler

 ---> System.Collections.Generic.KeyNotFoundException: The given key '25185' was not present in the dictionary.

   at System.Collections.Generic.Dictionary`2.get_Item(TKey key)

   at MySql.Data.MySqlClient.MySqlField.SetFieldEncoding()

   at MySql.Data.MySqlClient.NativeDriver.GetColumnData(MySqlField field)

   at MySql.Data.MySqlClient.NativeDriver.GetColumnsData(MySqlField[] columns)

   at MySql.Data.MySqlClient.Driver.GetColumns(Int32 count)

   at MySql.Data.MySqlClient.ResultSet.LoadColumns(Int32 numCols)

   at MySql.Data.MySqlClient.ResultSet..ctor(Driver d, Int32 statementId, Int32 numCols)

   at MySql.Data.MySqlClient.Driver.NextResult(Int32 statementId, Boolean force)

   at MySql.Data.MySqlClient.MySqlDataReader.NextResult()

   at MySql.Data.MySqlClient.MySqlDataReader.Close()

   at MySql.Data.MySqlClient.MySqlCommand.ResetReader()

   at MySql.Data.MySqlClient.MySqlCommand.ExecuteReader(CommandBehavior behavior)

   at MySql.Data.MySqlClient.MySqlCommand.ExecuteReader()

   at Rebus.MySql.Extensions.MySqlConnection.GetTableNames(MySqlConnection connection)

   at Rebus.MySql.Transport.MySqlTransport.CreateSchema()

   at Rebus.MySql.AsyncHelpers.CustomSynchronizationContext.<Run>b__7_0(Object _)

   at Rebus.MySql.AsyncHelpers.CustomSynchronizationContext.Run()

   at Rebus.MySql.AsyncHelpers.RunSync(Func`1 task)

   at Rebus.MySql.Transport.MySqlTransport.EnsureTableIsCreated()

   at Rebus.Config.MySqlTransportConfigurationExtension.<>c__DisplayClass2_0.<Configure>b__0(IResolutionContext context)

   at Rebus.Injection.Injectionist.Resolver`1.InvokeResolver(IResolutionContext context)

   at Rebus.Injection.Injectionist.ResolutionContext.Get[TService]()

   --- End of inner exception stack trace ---

   at Rebus.Injection.Injectionist.ResolutionContext.Get[TService]()

   at Rebus.Config.RebusConfigurer.<>c.<Start>b__12_11(IResolutionContext c)

   at Rebus.Injection.Injectionist.Resolver`1.InvokeResolver(IResolutionContext context)

   at Rebus.Injection.Injectionist.ResolutionContext.Get[TService]()

   at Rebus.Config.RebusConfigurer.<Start>b__12_22(IResolutionContext c)

   at Rebus.Injection.Injectionist.Resolver`1.InvokeResolver(IResolutionContext context)

   at Rebus.Injection.Injectionist.ResolutionContext.Get[TService]()

   at Rebus.Config.RebusConfigurer.<>c__DisplayClass12_1.<Start>b__23(IResolutionContext c)

   at Rebus.Injection.Injectionist.Resolver`1.InvokeResolver(IResolutionContext context)

   at Rebus.Injection.Injectionist.ResolutionContext.Get[TService]()

   at Rebus.Injection.Injectionist.Get[TService]()

   at Rebus.Config.RebusConfigurer.Start()

   at Rebus.ServiceProvider.ServiceCollectionExtensions.<>c__DisplayClass1_0.<AddRebus>b__4(IServiceProvider provider)

   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitFactory(FactoryCallSite factoryCallSite, RuntimeResolverContext context)

   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSiteMain(ServiceCallSite callSite, TArgument argument)

   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitCache(ServiceCallSite callSite, RuntimeResolverContext context, ServiceProviderEngineScope serviceProviderEngine, RuntimeResolverLock lockType)

   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitRootCache(ServiceCallSite singletonCallSite, RuntimeResolverContext context)

   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(ServiceCallSite callSite, TArgument argument)

   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.Resolve(ServiceCallSite callSite, ServiceProviderEngineScope scope)

   at Microsoft.Extensions.DependencyInjection.ServiceLookup.DynamicServiceProviderEngine.<>c__DisplayClass1_0.<RealizeService>b__0(ServiceProviderEngineScope scope)

   at Microsoft.Extensions.DependencyInjection.ServiceLookup.ServiceProviderEngine.GetService(Type serviceType, ServiceProviderEngineScope serviceProviderEngineScope)

   at Microsoft.Extensions.DependencyInjection.ServiceLookup.ServiceProviderEngineScope.GetService(Type serviceType)

   at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService(IServiceProvider provider, Type serviceType)

   at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService[T](IServiceProvider provider)

   at Rebus.ServiceProvider.ServiceProviderExtensions.UseRebus(IServiceProvider provider, Action`1 busAction)

If I delete and recreate the database table (empty) the "missing key" changes to, but the tables aren't created:

System.Collections.Generic.KeyNotFoundException: The given key '21510' was not present in the dictionary.

The live site is using Rebus.MySql@1.1.1 and Rebus.ServiceProvider@4.0.2 but I seem to get the same errors with current versions.

Configure Services looks like:

            services.AddRebus(configure => configure
                .Logging(l => l.Console(LogLevel.Debug))
                .Subscriptions(x => x.StoreInMySql(connectionString, "Subscriptions", true, true))
                .Transport(t => t.UseMySql(connectionString, "Transport", Configuration["AppName"])));

            services.AutoRegisterHandlersFromAssembly(GetType().Assembly.FullName);

Nothing has changed in the apps or infrastructure, just a restart of the service apps leads to the errors. I'm sorry for not having more useful detail but I just can't figure it out! :)

[mookid8000]

I think this looks more like a MySQL/MySql.Data problem than a Rebus problem. 🙂

A quick bingle for "mysql.data KeyNotFoundException" led me to https://stackoverflow.com/questions/24490164/keynotfoundexception-on-mysql-query-c-sharp which hints at the possibility that this could be caused by a collation mismatch.

As I am fairly certain that this has got nothing to do with Rebus/Rebus.MySql I am closing the issue now. Please don't hesitate to continue the thread here if you need me for anything Rebus-related.

[sbmw]

You're absolutely right, this was an external issue caused by a change to the hosted database. I needed to update all DB connectivity libraries.

Sorry for the panic issue and thank you very much for pointing me in the right direction!!

[mookid8000]

Happy to hear you got it fixed 🙂 👍