search

dotnet / efcore

EF Core is a modern object-database mapper for .NET. It supports LINQ queries, change tracking, updates, and schema migrations.
https://docs.microsoft.com/ef/
MIT License
13.8k stars 3.2k forks source link

EF9: Unexpected unbound variable error with closure #35152

Open BladeWise opened 2 days ago

BladeWise commented 2 days ago

Using EFCore 9 the following code (which uses a closure in an order-by condition)

static async Task OrderByClosureInt(SampleDbContext dbContext)
{
    int v = 1;
    Expression<Func<Dummy, object>> f = x => v;
    var q = dbContext.Set<Dummy>()
                     .OrderBy(f);

    await q.FirstOrDefaultAsync();
}

throws at runtime with the following error:

System.InvalidOperationException: An exception was thrown while attempting to evaluate the LINQ query parameter expression 'x => Convert(__v_0, Object)'. See the inner exception for more information.
       ---> System.InvalidOperationException: unbound variable: __v_0
Stacktrace ```Shell System.InvalidOperationException: An exception was thrown while attempting to evaluate the LINQ query parameter expression 'x => Convert(__v_0, Object)'. See the inner exception for more information. ---> System.InvalidOperationException: unbound variable: __v_0 at System.Linq.Expressions.Interpreter.LightCompiler.EnsureAvailableForClosure(ParameterExpression expr) at System.Linq.Expressions.Interpreter.LightCompiler.CompileQuoteUnaryExpression(Expression expr) at System.Linq.Expressions.Interpreter.LightCompiler.Compile(Expression expr) at System.Linq.Expressions.Interpreter.LightCompiler.CompileConvertUnaryExpression(Expression expr) at System.Linq.Expressions.Interpreter.LightCompiler.Compile(Expression expr) at System.Linq.Expressions.Interpreter.LightCompiler.CompileTop(LambdaExpression node) at System.Linq.Expressions.Expression`1.Compile(Boolean preferInterpretation) at Microsoft.EntityFrameworkCore.Query.Internal.ExpressionTreeFuncletizer.g__EvaluateCore|70_0(Expression expression, String& parameterName, Boolean& isContextAccessor) --- End of inner exception stack trace --- at Microsoft.EntityFrameworkCore.Query.Internal.ExpressionTreeFuncletizer.g__EvaluateCore|70_0(Expression expression, String& parameterName, Boolean& isContextAccessor) at Microsoft.EntityFrameworkCore.Query.Internal.ExpressionTreeFuncletizer.Evaluate(Expression expression, String& parameterName, Boolean& isContextAccessor) at Microsoft.EntityFrameworkCore.Query.Internal.ExpressionTreeFuncletizer.ProcessEvaluatableRoot(Expression evaluatableRoot, State& state, Boolean forceEvaluation) at Microsoft.EntityFrameworkCore.Query.Internal.ExpressionTreeFuncletizer.EvaluateList(IReadOnlyList`1 expressions, State[] expressionStates, List`1& children, Func`2 pathFromParentGenerator) at Microsoft.EntityFrameworkCore.Query.Internal.ExpressionTreeFuncletizer.VisitMethodCall(MethodCallExpression methodCall) at Microsoft.EntityFrameworkCore.Query.Internal.ExpressionTreeFuncletizer.Visit(Expression expression) at Microsoft.EntityFrameworkCore.Query.Internal.ExpressionTreeFuncletizer.Visit[T](ReadOnlyCollection`1 expressions, Func`2 elementVisitor, StateType& aggregateStateType, State[]& expressionStates, Boolean poolExpressionStates) at Microsoft.EntityFrameworkCore.Query.Internal.ExpressionTreeFuncletizer.Visit(ReadOnlyCollection`1 expressions, StateType& aggregateStateType, State[]& expressionStates, Boolean poolExpressionStates) at Microsoft.EntityFrameworkCore.Query.Internal.ExpressionTreeFuncletizer.VisitMethodCall(MethodCallExpression methodCall) at Microsoft.EntityFrameworkCore.Query.Internal.ExpressionTreeFuncletizer.Visit(Expression expression, State& state) at Microsoft.EntityFrameworkCore.Query.Internal.ExpressionTreeFuncletizer.ExtractParameters(Expression expression, IParameterValues parameterValues, Boolean parameterize, Boolean clearParameterizedValues, Boolean precompiledQuery, IReadOnlySet`1& nonNullableReferenceTypeParameters) at Microsoft.EntityFrameworkCore.Query.Internal.ExpressionTreeFuncletizer.ExtractParameters(Expression expression, IParameterValues parameterValues, Boolean parameterize, Boolean clearParameterizedValues) at Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler.ExtractParameters(Expression query, IParameterValues parameterValues, IDiagnosticsLogger`1 logger, Boolean compiledQuery, Boolean generateContextAccessors) at Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler.ExecuteCore[TResult](Expression query, Boolean async, CancellationToken cancellationToken) at Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler.ExecuteAsync[TResult](Expression query, CancellationToken cancellationToken) at Microsoft.EntityFrameworkCore.Query.Internal.EntityQueryProvider.ExecuteAsync[TResult](Expression expression, CancellationToken cancellationToken) at Microsoft.EntityFrameworkCore.EntityFrameworkQueryableExtensions.ExecuteAsync[TSource,TResult](MethodInfo operatorMethodInfo, IQueryable`1 source, Expression expression, CancellationToken cancellationToken) at Microsoft.EntityFrameworkCore.EntityFrameworkQueryableExtensions.ExecuteAsync[TSource,TResult](MethodInfo operatorMethodInfo, IQueryable`1 source, CancellationToken cancellationToken) at Microsoft.EntityFrameworkCore.EntityFrameworkQueryableExtensions.FirstOrDefaultAsync[TSource](IQueryable`1 source, CancellationToken cancellationToken) at Program.<
$>g__OrderByClosureInt|0_2(SampleDbContext dbContext) in C:\repos\EFIssueRepo\EFIssueRepo\Program.cs:line 67 at Program.
$(String[] args) in C:\repos\EFIssueRepo\EFIssueRepo\Program.cs:line 52 ```

The same code completes successfully using EFCore 8. Moreover, if the boxing is moved out of the expression like below

static async Task OrderByClosureInt(SampleDbContext dbContext)
{
    object v = 1;
    Expression<Func<Dummy, object>> f = x => v;
    var q = dbContext.Set<Dummy>()
                     .OrderBy(f);

    await q.FirstOrDefaultAsync();
}

the queryable is interpreted successfully.

The database provider is not relevant, because the exception is thrown at interpretation time (nevertheless, I have tested with both InMemory and PostgreSQL).

I have created a gist to reproduce the issue

Gist output ```Shell dbug: Microsoft.EntityFrameworkCore.Infrastructure[10401] An 'IServiceProvider' was created for internal use by Entity Framework. warn: Microsoft.EntityFrameworkCore.Model.Validation[10400] Sensitive data logging is enabled. Log entries and exception messages may include sensitive application data; this mode should only be enabled during development. dbug: Microsoft.EntityFrameworkCore.Infrastructure[10403] Entity Framework Core 9.0.0 initialized 'SampleDbContext' using provider 'Microsoft.EntityFrameworkCore.InMemory:9.0.0' with options: SensitiveDataLoggingEnabled StoreName=test dbug: Microsoft.EntityFrameworkCore.ChangeTracking[10806] Context 'SampleDbContext' started tracking 'Dummy' entity with key '{Id: 1}'. dbug: Microsoft.EntityFrameworkCore.Update[10004] SaveChanges starting for 'SampleDbContext'. dbug: Microsoft.EntityFrameworkCore.ChangeTracking[10800] DetectChanges starting for 'SampleDbContext'. dbug: Microsoft.EntityFrameworkCore.ChangeTracking[10801] DetectChanges completed for 'SampleDbContext'. info: Microsoft.EntityFrameworkCore.Update[30100] Saved 1 entities to in-memory store. dbug: Microsoft.EntityFrameworkCore.ChangeTracking[10807] The 'Dummy' entity with key '{Id: 1}' tracked by 'SampleDbContext' changed state from 'Added' to 'Unchanged'. dbug: Microsoft.EntityFrameworkCore.Update[10005] SaveChanges completed for 'SampleDbContext' with 1 entities written to the database. info: Program[0] Executing test OrderByClosureInt... fail: Program[0] Test OrderByClosureInt failed System.InvalidOperationException: An exception was thrown while attempting to evaluate the LINQ query parameter expression 'x => Convert(__v_0, Object)'. See the inner exception for more information. ---> System.InvalidOperationException: unbound variable: __v_0 at System.Linq.Expressions.Interpreter.LightCompiler.EnsureAvailableForClosure(ParameterExpression expr) at System.Linq.Expressions.Interpreter.LightCompiler.CompileQuoteUnaryExpression(Expression expr) at System.Linq.Expressions.Interpreter.LightCompiler.Compile(Expression expr) at System.Linq.Expressions.Interpreter.LightCompiler.CompileConvertUnaryExpression(Expression expr) at System.Linq.Expressions.Interpreter.LightCompiler.Compile(Expression expr) at System.Linq.Expressions.Interpreter.LightCompiler.CompileTop(LambdaExpression node) at System.Linq.Expressions.Expression`1.Compile(Boolean preferInterpretation) at Microsoft.EntityFrameworkCore.Query.Internal.ExpressionTreeFuncletizer.g__EvaluateCore|70_0(Expression expression, String& parameterName, Boolean& isContextAccessor) --- End of inner exception stack trace --- at Microsoft.EntityFrameworkCore.Query.Internal.ExpressionTreeFuncletizer.g__EvaluateCore|70_0(Expression expression, String& parameterName, Boolean& isContextAccessor) at Microsoft.EntityFrameworkCore.Query.Internal.ExpressionTreeFuncletizer.Evaluate(Expression expression, String& parameterName, Boolean& isContextAccessor) at Microsoft.EntityFrameworkCore.Query.Internal.ExpressionTreeFuncletizer.ProcessEvaluatableRoot(Expression evaluatableRoot, State& state, Boolean forceEvaluation) at Microsoft.EntityFrameworkCore.Query.Internal.ExpressionTreeFuncletizer.EvaluateList(IReadOnlyList`1 expressions, State[] expressionStates, List`1& children, Func`2 pathFromParentGenerator) at Microsoft.EntityFrameworkCore.Query.Internal.ExpressionTreeFuncletizer.VisitMethodCall(MethodCallExpression methodCall) at Microsoft.EntityFrameworkCore.Query.Internal.ExpressionTreeFuncletizer.Visit(Expression expression) at Microsoft.EntityFrameworkCore.Query.Internal.ExpressionTreeFuncletizer.Visit[T](ReadOnlyCollection`1 expressions, Func`2 elementVisitor, StateType& aggregateStateType, State[]& expressionStates, Boolean poolExpressionStates) at Microsoft.EntityFrameworkCore.Query.Internal.ExpressionTreeFuncletizer.Visit(ReadOnlyCollection`1 expressions, StateType& aggregateStateType, State[]& expressionStates, Boolean poolExpressionStates) at Microsoft.EntityFrameworkCore.Query.Internal.ExpressionTreeFuncletizer.VisitMethodCall(MethodCallExpression methodCall) at Microsoft.EntityFrameworkCore.Query.Internal.ExpressionTreeFuncletizer.Visit(Expression expression, State& state) at Microsoft.EntityFrameworkCore.Query.Internal.ExpressionTreeFuncletizer.ExtractParameters(Expression expression, IParameterValues parameterValues, Boolean parameterize, Boolean clearParameterizedValues, Boolean precompiledQuery, IReadOnlySet`1& nonNullableReferenceTypeParameters) at Microsoft.EntityFrameworkCore.Query.Internal.ExpressionTreeFuncletizer.ExtractParameters(Expression expression, IParameterValues parameterValues, Boolean parameterize, Boolean clearParameterizedValues) at Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler.ExtractParameters(Expression query, IParameterValues parameterValues, IDiagnosticsLogger`1 logger, Boolean compiledQuery, Boolean generateContextAccessors) at Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler.ExecuteCore[TResult](Expression query, Boolean async, CancellationToken cancellationToken) at Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler.ExecuteAsync[TResult](Expression query, CancellationToken cancellationToken) at Microsoft.EntityFrameworkCore.Query.Internal.EntityQueryProvider.ExecuteAsync[TResult](Expression expression, CancellationToken cancellationToken) at Microsoft.EntityFrameworkCore.EntityFrameworkQueryableExtensions.ExecuteAsync[TSource,TResult](MethodInfo operatorMethodInfo, IQueryable`1 source, Expression expression, CancellationToken cancellationToken) at Microsoft.EntityFrameworkCore.EntityFrameworkQueryableExtensions.ExecuteAsync[TSource,TResult](MethodInfo operatorMethodInfo, IQueryable`1 source, CancellationToken cancellationToken) at Microsoft.EntityFrameworkCore.EntityFrameworkQueryableExtensions.FirstOrDefaultAsync[TSource](IQueryable`1 source, CancellationToken cancellationToken) at Program.<
$>g__OrderByClosureInt|0_2(SampleDbContext dbContext) in C:\repos\EFIssueRepo\EFIssueRepo\Program.cs:line 67 at Program.
$(String[] args) in C:\repos\EFIssueRepo\EFIssueRepo\Program.cs:line 52 info: Program[0] Executing test OrderByClosureObject... dbug: Microsoft.EntityFrameworkCore.Query[10111] Compiling query expression: 'DbSet() .OrderBy(x => __v_0) .FirstOrDefault()' dbug: Microsoft.EntityFrameworkCore.Query[10107] Generated query execution expression: 'queryContext => ShapedQueryCompilingExpressionVisitor.SingleOrDefaultAsync( asyncEnumerable: new QueryingEnumerable( queryContext, new ResultEnumerable(() => InMemoryShapedQueryCompilingExpressionVisitor.Table( queryContext: queryContext, entityType: EntityType: Dummy) .OrderBy(valueBuffer => InMemoryExpressionTranslatingExpressionVisitor.GetParameterValue( queryContext: queryContext, parameterName: "__v_0")) .Select(valueBuffer => new ValueBuffer(new object[]{ (object)ExpressionExtensions.ValueBufferTryReadValue( valueBuffer: valueBuffer, index: 0, property: Property: Dummy.Id (int) Required PK AfterSave:Throw ValueGenerated.OnAdd) })) .FirstOrDefault()), Func, SampleDbContext, False, True ), cancellationToken: queryContext.CancellationToken)' info: Program[0] Test OrderByClosureObject succeeded dbug: Microsoft.EntityFrameworkCore.Infrastructure[10407] 'SampleDbContext' disposed. ```
roji commented 1 day ago

Confirmed regression from 8.0 to 9.0 - another thing to come out of the funcletizer rewrite, most likely.

Repro ```c# await using var context = new BlogContext(); await context.Database.EnsureDeletedAsync(); await context.Database.EnsureCreatedAsync(); int v = 1; Expression> f = x => v; var q = await context.Set().OrderBy(f).ToListAsync(); public class BlogContext : DbContext { public DbSet Blogs { get; set; } protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder) => optionsBuilder .UseSqlServer("Server=localhost;Database=test;User=SA;Password=Abcd5678;Connect Timeout=60;ConnectRetryCount=0;Encrypt=false") .LogTo(Console.WriteLine, LogLevel.Information) .EnableSensitiveDataLogging(); } public class Blog { public int Id { get; set; } public string Name { get; set; } } ```
roji commented 20 hours ago

The issue isn't the Expression variable, it's the convert-to-Object node that's added inside; the same error happens with:

var v = 1;
var q = await context.Set<Blog>().OrderBy(x => (object)v).ToListAsync();

This means that it's possible to work around this simply by having the lambda return the exact type, so no up-cast is needed:

Expression<Func<Dummy, int>> f = x => v;

The problem is that when we process the body of a lambda in the funcletizer, we incorrectly leave the state as EvaluatableWithCapturedVariables, after we've already evaluated the body inside; this causes the whole lambda to be evaluated again (technically the wrapping Quote node), and the attempt to run the LINQ interpreter over the ParameterExpression inside causes the exception. The correct state after evaluating something is NotEvaluatable, so that we don't attempt to re-evaluate something that already has been evaluated.

Note that the issue no longer reproes after recent switch to QueryParameterExpression in 10.0 (#35101), but for the reasons aren't material to this issue: the evaluated node is now a QueryParameterExpression rather than a ParameterExpression, and the LINQ interpreter surprisingly doesn't error on that unknown node type. But the bug is still there and could probably manifest in other ways.

BladeWise commented 18 hours ago

@roji I noticed that the issue was with conversion and applied the exact workaround you proposed. Thanks for sharing the actual root cause!

  • © Githubissues.
  • Githubissues is a development platform for aggregating issues.