Entity Exception -- provider failed to open

[kylegalbraith]

Not sure if I have something misconfigured (this is most likely the case) or if there is a bug somewhere else. But I make a call to GetByID in order to fetch the users account, however, every now and again I will get the following exception:

"name": "EntityException",
  "message": "The underlying provider failed on Open.",
  "source": "EntityFramework",
  "site": "Open",
  "stack": [
    {
      "type": "BrockAllen.MembershipReboot.EventBusUserAccountRepository`1",
      "method": "GetByID",
      "file": "c:\\ballen\\github\\brockallen\\BrockAllen.MembershipReboot\\src\\BrockAllen.MembershipReboot\\Repository\\EventBusUserAccountRepository.cs",
      "line": "77"
    }

Any clues on what I may be doing wrong or if there is an issue somewhere else?

[brockallen]

Hmm, really not sure.

[kylegalbraith]

This is still a randomly occurring incident that I cannot seem to nail down. Below is a more detailed stack trace that shows the error bubbling up through the EventBusUserAccountRespository.cs. The connection string is correctly in the web.config and the api that is kicking this off is a normal Web Api 2, in fact it isn't even async or anything like that.

"name": "EntityException",
  "message": "The underlying provider failed on Open.",
  "source": "EntityFramework",
  "site": "Open",
  "stack": [
    "   at System.Data.Entity.Core.EntityClient.EntityConnection.Open()",
    "   at System.Data.Entity.Core.Objects.ObjectContext.EnsureConnection(Boolean shouldMonitorTransactions)",
    "   at System.Data.Entity.Core.Objects.ObjectContext.ExecuteInTransaction[T](Func`1 func, IDbExecutionStrategy executionStrategy, Boolean startLocalTransaction, Boolean releaseConnectionOnSuccess)",
    "   at System.Data.Entity.Core.Objects.ObjectQuery`1.<>c__DisplayClass7.<GetResults>b__5()",
    "   at System.Data.Entity.SqlServer.DefaultSqlExecutionStrategy.Execute[TResult](Func`1 operation)",
    "   at System.Data.Entity.Core.Objects.ObjectQuery`1.GetResults(Nullable`1 forMergeOption)",
    "   at System.Data.Entity.Core.Objects.ObjectQuery`1.<System.Collections.Generic.IEnumerable<T>.GetEnumerator>b__0()",
    "   at System.Data.Entity.Internal.LazyEnumerator`1.MoveNext()",
    "   at System.Linq.Enumerable.SingleOrDefault[TSource](IEnumerable`1 source)",
    "   at System.Linq.Queryable.SingleOrDefault[TSource](IQueryable`1 source, Expression`1 predicate)",
    "   at BrockAllen.MembershipReboot.EventBusUserAccountRepository`1.GetByID(Guid id) in c:\\ballen\\github\\brockallen\\BrockAllen.MembershipReboot\\src\\BrockAllen.MembershipReboot\\Repository\\EventBusUserAccountRepository.cs:line 77",
    "   at BrockAllen.MembershipReboot.UserAccountService`1.GetByID(Guid id) in c:\\ballen\\github\\brockallen\\BrockAllen.MembershipReboot\\src\\BrockAllen.MembershipReboot\\AccountService\\UserAccountService.cs:line 265",

[brockallen]

Maybe it's just a DB failure?

[kylegalbraith]

I tried upgrading the db and even added MultipleActiveResultSet and yet I still get this failure. The innerException looks like this

"innerException": {
    "name": "InvalidOperationException",
    "message": "The connection was not closed. The connection's current state is connecting.",
    "source": "System.Data",
    "site": "TryOpenConnection",
    "stack": [
      "   at System.Data.ProviderBase.DbConnectionClosedConnecting.TryOpenConnection(DbConnection outerConnection, DbConnectionFactory connectionFactory, TaskCompletionSource`1 retry, DbConnectionOptions userOptions)",
      "   at System.Data.SqlClient.SqlConnection.TryOpenInner(TaskCompletionSource`1 retry)",
      "   at System.Data.SqlClient.SqlConnection.TryOpen(TaskCompletionSource`1 retry)",
      "   at System.Data.SqlClient.SqlConnection.Open()",
      "   at System.Data.Entity.Infrastructure.Interception.InternalDispatcher`1.Dispatch[TTarget,TInterceptionContext](TTarget target, Action`2 operation, TInterceptionContext interceptionContext, Action`3 executing, Action`3 executed)",
      "   at System.Data.Entity.Infrastructure.Interception.DbConnectionDispatcher.Open(DbConnection connection, DbInterceptionContext interceptionContext)",
      "   at System.Data.Entity.SqlServer.DefaultSqlExecutionStrategy.<>c__DisplayClass1.<Execute>b__0()",
      "   at System.Data.Entity.SqlServer.DefaultSqlExecutionStrategy.Execute[TResult](Func`1 operation)",
      "   at System.Data.Entity.Core.EntityClient.EntityConnection.Open()"
    ],

An example of where I am making this call and perhaps someone can spot my mistake...

if (userGuids == null || userGuids.Length == 0)
{
     throw new ArgumentException("Invalid user identifiers");
}

List<UserInfo> returnList = new List<UserInfo>();
foreach (Guid userGuid in userGuids)
{
     CustomUser account = _service.GetByID(userGuid);

     if (account == null)
     {
         throw new ArgumentException(string.Format("{0} is invalid", userGuid));
      }

     returnList.Add(CustomUserToUserInfo(account));
}

[brockallen]

It always fails from this same place?

[kylegalbraith]

Yes, but it does not fail every time.

[brockallen]

I guess I'd wonder how the DbContext is configured in DI -- per-instance, or per-http-request?

From the call stack, it's hard to say if it's anything in MR, or from EF, or from SqlServer...

[kylegalbraith]

I believe we are thinking all the same lines. This is setup in web api 2 via a startup class, below is the code snippet.

protected virtual void SetupServices(HttpConfiguration config, IAppBuilder app)
{
    CustomDatabase db = new CustomDatabase("MembershipReboot");

    CustomUserRepository cup = new CustomUserRepository(db);
    var websiteUrl = ConfigurationManager.AppSettings["websiteForEmails"];
    var emailFormatter = EmailMessageConfiguration.EmailConfig(app, websiteUrl);

    CustomConfig cc = new CustomConfig()
    {
        PasswordHashingIterationCount = 10000,
        RequireAccountVerification = false,
        EmailIsUsername = true,
        VerificationKeyLifetime = new TimeSpan(1, 0, 0, 0)
    };

    cc.AddEventHandler(new EmailAccountEventsHandler<CustomUser>(emailFormatter, new EmailMessageDelivery(Log)));
    CustomUserAccountService accountService = new CustomUserAccountService(cc, cup);
    config.Properties[ServiceNames.UserService] = new UserService(accountService);
}

[brockallen]

This looks like you're doing all this setup per-request, which seems reasonable. None of the MR code is designed to be multi-threaded. Out of curiosity, when is the DbContext disposed?

[kylegalbraith]

Well this is happening in the startup class of a web api 2 project, so I think it is a one time deal instead of request based, no? I was under the impression that the disposal of the context was handled in MR based on this...

public class CustomDatabase : MembershipRebootDbContext<CustomUser, CustomGroup>
{
    public CustomDatabase(string name)
        : base(name)
    {
    }
}

[brockallen]

I don't recall, to be honest, when it's disposed. I know there was a PR at some point to allow the mechanics to be configurable.

So looking thru the code, I don't see anywhere the ctx is disposed -- that's normally the job of the DI layer.

[ckieran]

@kylegalbraith FYI in case this helps, I have seen errors like this when two threads are re-using the same DB context and are calling Find at the same time.

@brockallen is correct - you should setup dbContext to be instance per http request in your DI config. Then within a request try not to use any multi-threaded code as an EntityConnection is not guaranteed to be threadsafe.

Happened upon this issue via a google search as I just added a Parallel.ForEach to my code to do 4 IO bound operations in parallel which ends up calling .Find() in my repo in 4 separate threads that resolve the same dbContext from DI .... which now reliably throws the above exception... :+1:

Hope you guys resolved your issues!