Describe the bug
Periodically I run into 500's caused by:
System.InvalidOperationException: A second operation was started on this context instance before a previous operation completed. This is usually caused by different threads concurrently using the same instance of DbContext.
I've seen this error in two different places in the UI and have observed different stack traces.
1) When clicking the new "Add My Projects" to org button
I've observed multiple different stack traces. It seems likely that those different stack traces represent a race condition that's triggering this.
at Microsoft.EntityFrameworkCore.Infrastructure.Internal.ConcurrencyDetector.EnterCriticalSection()
at Microsoft.EntityFrameworkCore.Query.Internal.SingleQueryingEnumerable`1.AsyncEnumerator.MoveNextAsync()
at Microsoft.EntityFrameworkCore.Query.ShapedQueryCompilingExpressionVisitor.SingleOrDefaultAsync[TSource](IAsyncEnumerable`1 asyncEnumerable, CancellationToken cancellationToken)
at Microsoft.EntityFrameworkCore.Query.ShapedQueryCompilingExpressionVisitor.SingleOrDefaultAsync[TSource](IAsyncEnumerable`1 asyncEnumerable, CancellationToken cancellationToken)
at LexBoxApi.Services.ProjectService.LookupProjectConfidentiality(Guid projectId) in /src/backend/LexBoxApi/Services/ProjectService.cs:line 201
at LexBoxApi.Services.PermissionService.CanViewProjectMembers(Guid projectId) in /src/backend/LexBoxApi/Services/PermissionService.cs:line 102
at LexBoxApi.GraphQL.CustomTypes.ProjectMembersVisibilityMiddleware.InvokeAsync(IMiddlewareContext context, IPermissionService permissionService, LoggedInContext loggedInContext) in /src/backend/LexBoxApi/GraphQL/CustomTypes/ProjectMembersVisibilityMiddleware.cs:line 18
at HotChocolate.Utilities.MiddlewareCompiler`1.ExpressionHelper.AwaitTaskHelper(Task task)
at LexBoxApi.GraphQL.CustomTypes.RefreshJwtProjectMembershipMiddleware.InvokeAsync(IMiddlewareContext context) in /src/backend/LexBoxApi/GraphQL/CustomTypes/RefreshJwtProjectMembershipMiddleware.cs:line 12
at HotChocolate.Utilities.MiddlewareCompiler`1.ExpressionHelper.AwaitTaskHelper(Task task)
at DataAnnotatedModelValidations.Middleware.ValidatorMiddleware.InvokeAsync(IMiddlewareContext context)
at HotChocolate.Utilities.MiddlewareCompiler`1.ExpressionHelper.AwaitTaskHelper(Task task)
at HotChocolate.Execution.Processing.Tasks.ResolverTask.ExecuteResolverPipelineAsync(CancellationToken cancellationToken)
at HotChocolate.Execution.Processing.Tasks.ResolverTask.TryExecuteAsync(CancellationToken cancellationToken)
at Microsoft.EntityFrameworkCore.Query.Internal.SingleQueryingEnumerable`1.AsyncEnumerator.MoveNextAsync()
at Microsoft.EntityFrameworkCore.Query.ShapedQueryCompilingExpressionVisitor.SingleOrDefaultAsync[TSource](IAsyncEnumerable`1 asyncEnumerable, CancellationToken cancellationToken)
at Microsoft.EntityFrameworkCore.Query.ShapedQueryCompilingExpressionVisitor.SingleOrDefaultAsync[TSource](IAsyncEnumerable`1 asyncEnumerable, CancellationToken cancellationToken)
at LexBoxApi.Services.ProjectService.LookupProjectConfidentiality(Guid projectId) in /src/backend/LexBoxApi/Services/ProjectService.cs:line 201
at LexBoxApi.Services.PermissionService.CanViewProjectMembers(Guid projectId) in /src/backend/LexBoxApi/Services/PermissionService.cs:line 102
at LexBoxApi.GraphQL.CustomTypes.ProjectMembersVisibilityMiddleware.InvokeAsync(IMiddlewareContext context, IPermissionService permissionService, LoggedInContext loggedInContext) in /src/backend/LexBoxApi/GraphQL/CustomTypes/ProjectMembersVisibilityMiddleware.cs:line 18
at HotChocolate.Utilities.MiddlewareCompiler`1.ExpressionHelper.AwaitTaskHelper(Task task)
at LexBoxApi.GraphQL.CustomTypes.RefreshJwtProjectMembershipMiddleware.InvokeAsync(IMiddlewareContext context) in /src/backend/LexBoxApi/GraphQL/CustomTypes/RefreshJwtProjectMembershipMiddleware.cs:line 12
at HotChocolate.Utilities.MiddlewareCompiler`1.ExpressionHelper.AwaitTaskHelper(Task task)
at DataAnnotatedModelValidations.Middleware.ValidatorMiddleware.InvokeAsync(IMiddlewareContext context)
at HotChocolate.Utilities.MiddlewareCompiler`1.ExpressionHelper.AwaitTaskHelper(Task task)
at HotChocolate.Execution.Processing.Tasks.ResolverTask.ExecuteResolverPipelineAsync(CancellationToken cancellationToken)
at HotChocolate.Execution.Processing.Tasks.ResolverTask.TryExecuteAsync(CancellationToken cancellationToken)
at Microsoft.EntityFrameworkCore.Query.Internal.SingleQueryingEnumerable`1.AsyncEnumerator.MoveNextAsync()
at Microsoft.EntityFrameworkCore.Query.ShapedQueryCompilingExpressionVisitor.SingleOrDefaultAsync[TSource](IAsyncEnumerable`1 asyncEnumerable, CancellationToken cancellationToken)
at Microsoft.EntityFrameworkCore.Query.ShapedQueryCompilingExpressionVisitor.SingleOrDefaultAsync[TSource](IAsyncEnumerable`1 asyncEnumerable, CancellationToken cancellationToken)
at LexBoxApi.Auth.LexAuthService.RefreshUser(Guid userId, String updatedValue) in /src/backend/LexBoxApi/Auth/LexAuthService.cs:line 90
at LexBoxApi.GraphQL.CustomTypes.RefreshJwtProjectMembershipMiddleware.RefreshUser(IMiddlewareContext context, Guid userId) in /src/backend/LexBoxApi/GraphQL/CustomTypes/RefreshJwtProjectMembershipMiddleware.cs:line 68
at LexBoxApi.GraphQL.CustomTypes.RefreshJwtProjectMembershipMiddleware.InvokeAsync(IMiddlewareContext context) in /src/backend/LexBoxApi/GraphQL/CustomTypes/RefreshJwtProjectMembershipMiddleware.cs:line 51
at HotChocolate.Utilities.MiddlewareCompiler`1.ExpressionHelper.AwaitTaskHelper(Task task)
at DataAnnotatedModelValidations.Middleware.ValidatorMiddleware.InvokeAsync(IMiddlewareContext context)
at HotChocolate.Utilities.MiddlewareCompiler`1.ExpressionHelper.AwaitTaskHelper(Task task)
at HotChocolate.Execution.Processing.Tasks.ResolverTask.ExecuteResolverPipelineAsync(CancellationToken cancellationToken)
at HotChocolate.Execution.Processing.Tasks.ResolverTask.TryExecuteAsync(CancellationToken cancellationToken)
at Microsoft.EntityFrameworkCore.Query.Internal.SingleQueryingEnumerable`1.AsyncEnumerator.MoveNextAsync()
at Microsoft.EntityFrameworkCore.Query.ShapedQueryCompilingExpressionVisitor.SingleOrDefaultAsync[TSource](IAsyncEnumerable`1 asyncEnumerable, CancellationToken cancellationToken)
at Microsoft.EntityFrameworkCore.Query.ShapedQueryCompilingExpressionVisitor.SingleOrDefaultAsync[TSource](IAsyncEnumerable`1 asyncEnumerable, CancellationToken cancellationToken)
at LexBoxApi.Services.ProjectService.LookupProjectOrgIds(Guid projectId) in /src/backend/LexBoxApi/Services/ProjectService.cs:line 183
at LexBoxApi.Services.PermissionService.ManagesOrgThatOwnsProject(Guid projectId) in /src/backend/LexBoxApi/Services/PermissionService.cs:line 21
at LexBoxApi.Services.PermissionService.CanManageProject(Guid projectId) in /src/backend/LexBoxApi/Services/PermissionService.cs:line 112
at LexBoxApi.Services.PermissionService.CanViewProjectMembers(Guid projectId) in /src/backend/LexBoxApi/Services/PermissionService.cs:line 101
at LexBoxApi.GraphQL.CustomTypes.ProjectMembersVisibilityMiddleware.InvokeAsync(IMiddlewareContext context, IPermissionService permissionService, LoggedInContext loggedInContext) in /src/backend/LexBoxApi/GraphQL/CustomTypes/ProjectMembersVisibilityMiddleware.cs:line 18
at HotChocolate.Utilities.MiddlewareCompiler`1.ExpressionHelper.AwaitTaskHelper(Task task)
at LexBoxApi.GraphQL.CustomTypes.RefreshJwtProjectMembershipMiddleware.InvokeAsync(IMiddlewareContext context) in /src/backend/LexBoxApi/GraphQL/CustomTypes/RefreshJwtProjectMembershipMiddleware.cs:line 12
at HotChocolate.Utilities.MiddlewareCompiler`1.ExpressionHelper.AwaitTaskHelper(Task task)
at DataAnnotatedModelValidations.Middleware.ValidatorMiddleware.InvokeAsync(IMiddlewareContext context)
at HotChocolate.Utilities.MiddlewareCompiler`1.ExpressionHelper.AwaitTaskHelper(Task task)
at HotChocolate.Execution.Processing.Tasks.ResolverTask.ExecuteResolverPipelineAsync(CancellationToken cancellationToken)
at HotChocolate.Execution.Processing.Tasks.ResolverTask.TryExecuteAsync(CancellationToken cancellationToken)
2) When hovering over a project I'm in that is in an org I am not in
I think I've seen 2 different stack traces for this:
at Microsoft.EntityFrameworkCore.Query.Internal.SingleQueryingEnumerable`1.AsyncEnumerator.MoveNextAsync()
at Microsoft.EntityFrameworkCore.Query.ShapedQueryCompilingExpressionVisitor.SingleOrDefaultAsync[TSource](IAsyncEnumerable`1 asyncEnumerable, CancellationToken cancellationToken)
at Microsoft.EntityFrameworkCore.Query.ShapedQueryCompilingExpressionVisitor.SingleOrDefaultAsync[TSource](IAsyncEnumerable`1 asyncEnumerable, CancellationToken cancellationToken)
at LexBoxApi.Auth.LexAuthService.RefreshUser(Guid userId, String updatedValue) in /src/backend/LexBoxApi/Auth/LexAuthService.cs:line 90
at LexBoxApi.GraphQL.CustomTypes.RefreshJwtOrgMembershipMiddleware.RefreshUser(IMiddlewareContext context, Guid userId) in /src/backend/LexBoxApi/GraphQL/CustomTypes/RefreshJwtOrgMembershipMiddleware.cs:line 68
at LexBoxApi.GraphQL.CustomTypes.RefreshJwtOrgMembershipMiddleware.InvokeAsync(IMiddlewareContext context) in /src/backend/LexBoxApi/GraphQL/CustomTypes/RefreshJwtOrgMembershipMiddleware.cs:line 34
at HotChocolate.Utilities.MiddlewareCompiler`1.ExpressionHelper.AwaitTaskHelper(Task task)
at DataAnnotatedModelValidations.Middleware.ValidatorMiddleware.InvokeAsync(IMiddlewareContext context)
at HotChocolate.Utilities.MiddlewareCompiler`1.ExpressionHelper.AwaitTaskHelper(Task task)
at HotChocolate.Execution.Processing.Tasks.ResolverTask.ExecuteResolverPipelineAsync(CancellationToken cancellationToken)
at HotChocolate.Execution.Processing.Tasks.ResolverTask.TryExecuteAsync(CancellationToken cancellationToken)
at Microsoft.EntityFrameworkCore.Infrastructure.Internal.ConcurrencyDetector.EnterCriticalSection()
at Microsoft.EntityFrameworkCore.Query.Internal.SingleQueryingEnumerable`1.AsyncEnumerator.MoveNextAsync()
at Microsoft.EntityFrameworkCore.Query.ShapedQueryCompilingExpressionVisitor.SingleOrDefaultAsync[TSource](IAsyncEnumerable`1 asyncEnumerable, CancellationToken cancellationToken)
at Microsoft.EntityFrameworkCore.Query.ShapedQueryCompilingExpressionVisitor.SingleOrDefaultAsync[TSource](IAsyncEnumerable`1 asyncEnumerable, CancellationToken cancellationToken)
at LexBoxApi.Auth.LexAuthService.RefreshUser(Guid userId, String updatedValue) in /src/backend/LexBoxApi/Auth/LexAuthService.cs:line 90
at LexBoxApi.GraphQL.CustomTypes.RefreshJwtOrgMembershipMiddleware.RefreshUser(IMiddlewareContext context, Guid userId) in /src/backend/LexBoxApi/GraphQL/CustomTypes/RefreshJwtOrgMembershipMiddleware.cs:line 68
at LexBoxApi.GraphQL.CustomTypes.RefreshJwtOrgMembershipMiddleware.InvokeAsync(IMiddlewareContext context) in /src/backend/LexBoxApi/GraphQL/CustomTypes/RefreshJwtOrgMembershipMiddleware.cs:line 34
at HotChocolate.Utilities.MiddlewareCompiler`1.ExpressionHelper.AwaitTaskHelper(Task task)
at DataAnnotatedModelValidations.Middleware.ValidatorMiddleware.InvokeAsync(IMiddlewareContext context)
at HotChocolate.Utilities.MiddlewareCompiler`1.ExpressionHelper.AwaitTaskHelper(Task task)
at HotChocolate.Execution.Processing.Tasks.ResolverTask.ExecuteResolverPipelineAsync(CancellationToken cancellationToken)
at HotChocolate.Execution.Processing.Tasks.ResolverTask.TryExecuteAsync(CancellationToken cancellationToken)
To Reproduce
I don't know how to reproduce it.
Expected behavior
We shouldn't run into errors triggered by these EF concurrency exceptions.
myieye
commented
2 weeks ago
Describe the bug Periodically I run into 500's caused by:
System.InvalidOperationException: A second operation was started on this context instance before a previous operation completed. This is usually caused by different threads concurrently using the same instance of DbContext.I've seen this error in two different places in the UI and have observed different stack traces.
1) When clicking the new "Add My Projects" to org button
I've observed multiple different stack traces. It seems likely that those different stack traces represent a race condition that's triggering this.
Stack trace 1: (trace)
Stack trace 2: (trace)
Stack trace 3: (trace)
Stack trace 4: (trace)
2) When hovering over a project I'm in that is in an org I am not in
I think I've seen 2 different stack traces for this:
Stack trace 1: (trace)
Stack trace 2: (trace)
To Reproduce I don't know how to reproduce it.
Expected behavior We shouldn't run into errors triggered by these EF concurrency exceptions.