search

IntelliTect / Coalesce

Quickly build amazing web apps
https://intellitect.github.io/Coalesce/
Apache License 2.0
64 stars 22 forks source link

Odd interaction between Linq.Dynamic and EF Core breaks sorted List queries in rare cases. #25

Closed ascott18 closed 6 years ago

ascott18 commented 6 years ago

There is an issue in both EF Core & Linq.Dynamic (https://github.com/StefH/System.Linq.Dynamic.Core), but arguably Linq.Dynamic is more at fault than EF Core, since it is (in part) designed to work with EF Core, and EF Core works perfectly when constructing the same statement with a compiler-interpreted lambda.

The issue is as follows:

Given a model like the following (See comment below for a full repro project.):

class Context : DbContext {
    public DbSet<ParentEntity> ParentEntities { get; set; }
    public DbSet<RootEntity> RootEntities { get; set; }
    public DbSet<ChildEntity> ChildEntities { get; set; }
}
class ParentEntity {
    int ParentEntityId { get; set; }

    string Name { get;set; }
}
class RootEntity {
    int RootEntityId { get; set; }

    string Name { get;set; }

    int? ParentEntityId { get; set; }
    ParentEntity ParentEntity { get; set; }

    ICollection<ChildEntity> Children { get; set; }
}
class ChildEntity {
    int ChildEntityId { get; set; }

    int RootEntityId { get; set; }
    RootEntity RootEntity { get; set; }
}

The following will work:

var context = new Context();
context.RootEntities
    .Include(r => r.ParentEntity)
    .Include(r => r.Children)
    .OrderBy(r => r.ParentEntity == null ? "" : r.ParentEntity.Name).ThenBy(r => r.Name)

And the following will NOT work:

using System.Linq.Dynamic.Core;
var context = new Context();
context.RootEntities
    .Include(r => r.ParentEntity)
    .Include(r => r.Children)
    .OrderBy("ParentEntity == null ? \"\" : ParentEntity.Name ASC, Name ASC")

The exception thrown by the second case:

System.ArgumentNullException occurred
  HResult=0x80004003
  Message=Value cannot be null.
Parameter name: source
  Source=<Cannot evaluate the exception source>
  StackTrace:
   at System.Linq.Enumerable.Aggregate[TSource,TAccumulate](IEnumerable`1 source, TAccumulate seed, Func`3 func)
   at Microsoft.EntityFrameworkCore.Query.Internal.AnonymousObject.GetHashCode()
   at System.Collections.Generic.ObjectEqualityComparer`1.GetHashCode(T obj)
   at System.Linq.Set`1.InternalGetHashCode(TElement value)
   at System.Linq.Set`1.Add(TElement value)
   at System.Linq.AsyncEnumerable.DistinctAsyncIterator`1.<MoveNextCore>d__10.MoveNext()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Linq.AsyncEnumerable.AsyncIterator`1.<MoveNext>d__10.MoveNext()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Linq.Internal.Lookup`2.<CreateForJoinAsync>d__16.MoveNext()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Linq.AsyncEnumerable.JoinAsyncIterator`4.<MoveNextCore>d__20.MoveNext()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Linq.AsyncEnumerable.AsyncIterator`1.<MoveNext>d__10.MoveNext()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Linq.AsyncEnumerable.<Aggregate_>d__6`3.MoveNext()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Linq.OrderedAsyncEnumerable`2.<Initialize>d__10.MoveNext()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Linq.OrderedAsyncEnumerable`2.<Initialize>d__10.MoveNext()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Linq.OrderedAsyncEnumerable`2.<Initialize>d__10.MoveNext()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Linq.OrderedAsyncEnumerable`2.<MoveNextCore>d__9.MoveNext()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Linq.AsyncEnumerable.AsyncIterator`1.<MoveNext>d__10.MoveNext()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Linq.AsyncEnumerable.SelectEnumerableAsyncIterator`2.<MoveNextCore>d__7.MoveNext()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Linq.AsyncEnumerable.AsyncIterator`1.<MoveNext>d__10.MoveNext()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.EntityFrameworkCore.Query.Internal.QueryBuffer.<IncludeCollectionAsync>d__12.MoveNext()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.EntityFrameworkCore.Query.Internal.IncludeCompiler.<_IncludeAsync>d__18`1.MoveNext()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.EntityFrameworkCore.Query.ExpressionVisitors.Internal.TaskLiftingExpressionVisitor.<_ExecuteAsync>d__8`1.MoveNext()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.EntityFrameworkCore.Query.EntityQueryModelVisitor.AsyncSelectEnumerable`2.AsyncSelectEnumerator.<MoveNext>d__3.MoveNext()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Linq.AsyncEnumerable.SelectEnumerableAsyncIterator`2.<MoveNextCore>d__7.MoveNext()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Linq.AsyncEnumerable.AsyncIterator`1.<MoveNext>d__10.MoveNext()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.EntityFrameworkCore.Query.Internal.AsyncLinqOperatorProvider.ExceptionInterceptor`1.EnumeratorExceptionInterceptor.<MoveNext>d__5.MoveNext()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Linq.AsyncEnumerable.<Aggregate_>d__6`3.MoveNext()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult()
   at IntelliTect.Coalesce.Controllers.BaseApiController`3.<ListImplementation>d__25.MoveNext() in Coalesce\src\IntelliTect.Coalesce\Controllers\BaseApiController.cs:line 188

The expression generated by the correct, working query:

.Call System.Linq.Queryable.ThenBy(
    .Call System.Linq.Queryable.OrderBy(
        .Call Microsoft.EntityFrameworkCore.EntityFrameworkQueryableExtensions.Include(
            .Constant<Microsoft.EntityFrameworkCore.Query.Internal.EntityQueryable`1[RootEntity]>(Microsoft.EntityFrameworkCore.Query.Internal.EntityQueryable`1[RootEntity]),
            '(.Lambda #Lambda1<System.Func`2[RootEntity,System.Collections.Generic.List`1[ChildEntity]]>))
        ,
        '(.Lambda #Lambda2<System.Func`2[RootEntity,System.String]>)),
    '(.Lambda #Lambda3<System.Func`2[RootEntity,System.String]>))

.Lambda #Lambda1<System.Func`2[RootEntity,System.Collections.Generic.List`1[ChildEntity]]>(RootEntity $e)
{
    $e.Children
}

.Lambda #Lambda2<System.Func`2[RootEntity,System.String]>(RootEntity $r)
{
    .If ($r.ParentEntity == null) {
        ""
    } .Else {
        ($r.ParentEntity).Name
    }
}

.Lambda #Lambda3<System.Func`2[RootEntity,System.String]>(RootEntity $r)
{
    $r.Name
}

The expression generated by Linq.Dynamic that causes the error inside EF Core:

.Call System.Linq.Queryable.ThenBy(
    .Call System.Linq.Queryable.OrderBy(
        .Call Microsoft.EntityFrameworkCore.EntityFrameworkQueryableExtensions.Include(
            .Constant<Microsoft.EntityFrameworkCore.Query.Internal.EntityQueryable`1[RootEntity]>(Microsoft.EntityFrameworkCore.Query.Internal.EntityQueryable`1[RootEntity]),
            '(.Lambda #Lambda1<System.Func`2[RootEntity,System.Collections.Generic.List`1[ChildEntity]]>))
        ,
        '(.Lambda #Lambda2<System.Func`2[RootEntity,System.String]>)),
    '(.Lambda #Lambda3<System.Func`2[RootEntity,System.String]>))

.Lambda #Lambda1<System.Func`2[RootEntity,System.Collections.Generic.List`1[ChildEntity]]>(RootEntity $e)
{
    $e.Children
}

.Lambda #Lambda2<System.Func`2[RootEntity,System.String]>(RootEntity $var1)
{
    .If ((System.Object)$var1.ParentEntity == null) {
        ""
    } .Else {
        ($var1.ParentEntity).Name
    }
}

.Lambda #Lambda3<System.Func`2[RootEntity,System.String]>(RootEntity $var1)
{
    $var1.Name
}

EF Core is getting confused by the object cast here and going down some code path that eventually ends in an error. This issue ONLY happens when the child collection is included. If the include of the child collection is removed, no error occurs.

StefH commented 6 years ago

@ascott18 Can you provide details how to extract the generated expressions in above issue? (Or did you just use the debugger?)

ascott18 commented 6 years ago

@StefH I got it through the debugger (query.Expression.DebugView). Here's a full project that reproduces the issue:

QueryBug.zip

ascott18 commented 6 years ago

Here it is a bit cleaner. Also added another query to show the same behavior without using a dynamic expression: QueryBugCleanedUp.zip

finlaysonc commented 6 years ago

Here's an alternative using the Z.Expressions library.
QueryBug-ZExpressionAlternative.zip

    string expression = $@"OrderBy(r => r.ParentEntity == null ? """" : r.ParentEntity.Name).ThenBy(r => r.Name);";
    var dynamicQuery = context.RootEntities
        .Include(r => r.ParentEntity)
        .Include(r => r.Children)
        .Execute<IQueryable<RootEntity>>(
            expression);