linq2db / linq2db.EntityFrameworkCore

Bring power of Linq To DB to Entity Framework Core projects
MIT License
449 stars 39 forks source link

Linq2Db fails to translate recursive CTE with ef core TPH #375

Closed neistow closed 6 months ago

neistow commented 6 months ago

I'm trying to do the following query with linq2db and ef core:

with query as (
    select i.*
    from StorageItems i
    where Id = 'da7a4317-05ea-4756-8ce8-8e76f2cd7dee'
    union all
    select i.*
    from StorageItems i
             join query cte on cte.Id = i.ParentId
)
select *
from query

Here's my dbcontext configuration:

public class AppDbContext : DbContext
{
    public DbSet<StorageItem> StorageItems => Set<StorageItem>();

    protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
    {
        optionsBuilder.UseSqlServer("");
        optionsBuilder.UseLinqToDB();

        base.OnConfiguring(optionsBuilder);
    }

    protected override void OnModelCreating(ModelBuilder modelBuilder)
    {
        modelBuilder.Entity<StorageItem>()
            .HasDiscriminator(x => x.Type);

        base.OnModelCreating(modelBuilder);
    }
}
public abstract class StorageItem
{
    public Guid Id { get; set; }

    public StorageFolder? Parent { get; set; }
    public Guid? ParentId { get; set; }

    public abstract string Type { get; protected set; }
}

public class StorageFolder : StorageItem
{
    public string? Color { get; set; }
    public override string Type { get; protected set; } = "Folder";
}

public class StorageFile : StorageItem
{
    public long Size { get; set; }
    public override string Type { get; protected set; } = "File";
}

public class StorageEnhFile : StorageFile
{
    public bool SomeProp { get; set; }

    public override string Type { get; protected set; } = "EFile";
}

Here's translated version using linq2db:

var guid = Guid.Parse("84eb9daa-fbcf-464a-a9da-9b1fbc599a72");

var cte = _dbContext.StorageItems
    .ToLinqToDBTable().DataContext
    .GetCte<StorageItem>(storageItems =>
    {
        var q1 = storageItems.Where(x => x.Id == guid);
        var q2 = storageItems.SelectMany(
            x => storageItems.InnerJoin(cte => x.Id == cte.Id),
            (x, _) => x
        );
        return q1.Concat(q2);
    });

var res = await cte.ToListAsyncEF();

Exception I'm getting:

: Sequence contains no matching element
   at System.Linq.ThrowHelper.ThrowNoMatchException()
   at System.Linq.Enumerable.First[TSource](IEnumerable`1 source, Func`2 predicate)
   at LinqToDB.Linq.Builder.SetOperationBuilder.SetOperationContext.AddSequence(SubQueryContext sequence, SqlSetOperator setOperator)
   at LinqToDB.Linq.Builder.SetOperationBuilder.SetOperationContext..ctor(SubQueryContext sequence1, SubQueryContext sequence2, MethodCallExpression methodCall, SqlSetOperator setOperator)
   at LinqToDB.Linq.Builder.SetOperationBuilder.BuildMethodCall(ExpressionBuilder builder, MethodCallExpression methodCall, BuildInfo buildInfo)
   at LinqToDB.Linq.Builder.MethodCallBuilder.BuildSequence(ExpressionBuilder builder, BuildInfo buildInfo)
   at LinqToDB.Linq.Builder.ExpressionBuilder.BuildSequence(BuildInfo buildInfo)
   at LinqToDB.Linq.Builder.TableBuilder.<>c__DisplayClass11_0.<BuildCteContext>b__1(CteClause cteClause)
   at LinqToDB.Linq.Builder.ExpressionBuilder.BuildCte(Expression cteExpression, Func`2 buildFunc)
   at LinqToDB.Linq.Builder.TableBuilder.BuildCteContext(ExpressionBuilder builder, BuildInfo buildInfo)
   at LinqToDB.Linq.Builder.TableBuilder.BuildSequence(ExpressionBuilder builder, BuildInfo buildInfo)
   at LinqToDB.Linq.Builder.ExpressionBuilder.BuildSequence(BuildInfo buildInfo)
   at LinqToDB.Linq.Builder.ExpressionBuilder.Build[T]()
   at LinqToDB.Linq.Query`1.CreateQuery(ExpressionTreeOptimizationContext optimizationContext, ParametersContext parametersContext, IDataContext dataContext, Expression expr)
   at LinqToDB.Linq.Query`1.GetQuery(IDataContext dataContext, Expression& expr, Boolean& dependsOnParameters)
   at LinqToDB.Linq.ExpressionQuery`1.GetQuery(Expression& expression, Boolean cache, Boolean& dependsOnParameters)
   at LinqToDB.Linq.ExpressionQuery`1.GetAsyncEnumerator(CancellationToken cancellationToken)
   at System.Runtime.CompilerServices.ConfiguredCancelableAsyncEnumerable`1.GetAsyncEnumerator()
   at Microsoft.EntityFrameworkCore.EntityFrameworkQueryableExtensions.ToListAsync[TSource](IQueryable`1 source, CancellationToken cancellationToken)
   ....

I did a little bit of investigation and found out that this line errors, because it can't find memberinfo that has the same declaring type parent that union parameter type (which is StorageItem) has. Basically it fails because StorageItem is not convertible to StorageFolder, which is kind of logical, but EF is able to create concrete types and cast them to base class while querying. Not sure if it's a bug or just linq2db doesn't support TPH with recursive CTE

sdanyliv commented 6 months ago

Cte implementation in linq2db (should be fxed in upcoming version 6) do not support inheritance. As a workaround you can return PK and materialize joined values. Your query also totally incorrect and I have prepared correct version:

Define intermediate class

class StorageItemCte
{
    public Guid Id { get; set; }
}
var guid = Guid.Parse("84eb9daa-fbcf-464a-a9da-9b1fbc599a72");
using var db = _dbContext.CreateLinqToDBConnection();

var cteQuery = db.GetCte<StorageItemCte>(storageItems =>
    {
        return 
        (
            from si in _dbContext.StorageItems
            where si.Id == guid
            select new StorageItemCte { Id = si.Id }
        )
        .Concat
        (
            from si in _dbContext.StorageItems
            from cte in storageItems.InnerJoin(cte => cte.Id == si.ParentId)
            select new StorageItemCte { Id = si.Id }
        )
    });

var resultQuery = 
    from cte in cteQuery
    from si in _dbContext.StorageItems.InnerJoin(si => si.Id == cte.Id)
    select si;

var res = await result.ToListAsyncLinqToDB();
neistow commented 6 months ago

@sdanyliv Thanks for the quick reply.

Your query also totally incorrect

It should be, I copy-pasted it from docs and just used IDE to convert LINQ to method chain.

I have prepared correct version

Appreciate your help, that would be a valid workaround... unless TPH issues with linq2db again.

First of all, EF core doesn't really like the following way to define discriminator and can't materialize the final result since it doesn't know what "Folder" discriminator is:

modelBuilder.Entity<StorageItem>()
            .HasDiscriminator(x => x.Type);

Usually I define my discriminators through shadow properties, which works:

        modelBuilder.Entity<StorageItem>()
            .HasDiscriminator<string>("__type")
            .HasValue<StorageFolder>("Folder")
            .HasValue<StorageFile>("File")
            .HasValue<StorageEnhFile>("EFile");

...But it seems that linq2db doesn't know about shadow properties from EF, so it throws:

LinqToDB.Linq.LinqException: Inheritance Discriminator is not defined for the 'EfPlayground.Models.StorageItem' hierarchy.
   at LinqToDB.Mapping.EntityDescriptor.InitInheritanceMapping(Action`2 onEntityDescriptorCreated)
   at LinqToDB.Mapping.EntityDescriptor.Init(Action`2 onEntityDescriptorCreated)
   at LinqToDB.Mapping.EntityDescriptor..ctor(MappingSchema mappingSchema, Type type, Action`2 onEntityDescriptorCreated)
   at LinqToDB.Mapping.MappingSchema.<>c.<GetEntityDescriptor>b__98_0(ICacheEntry`2 o, ValueTuple`2 context)
   at LinqToDB.Common.Internal.Cache.CacheExtensions.GetOrCreate[TItem,TKey,TContext](IMemoryCache`2 cache, TKey key, TContext context, Func`3 factory)
   at LinqToDB.Mapping.MappingSchema.GetEntityDescriptor(Type type, Action`2 onEntityDescriptorCreated)
   at LinqToDB.Linq.Builder.TableBuilder.TableContext..ctor(ExpressionBuilder builder, BuildInfo buildInfo, Type originalType)
   at LinqToDB.Linq.Builder.TableBuilder.BuildSequence(ExpressionBuilder builder, BuildInfo buildInfo)
   at LinqToDB.Linq.Builder.ExpressionBuilder.BuildSequence(BuildInfo buildInfo)
   at LinqToDB.Linq.Builder.WhereBuilder.BuildMethodCall(ExpressionBuilder builder, MethodCallExpression methodCall, BuildInfo buildInfo)
   at LinqToDB.Linq.Builder.MethodCallBuilder.BuildSequence(ExpressionBuilder builder, BuildInfo buildInfo)
   at LinqToDB.Linq.Builder.ExpressionBuilder.BuildSequence(BuildInfo buildInfo)
   at LinqToDB.Linq.Builder.SelectBuilder.BuildMethodCall(ExpressionBuilder builder, MethodCallExpression methodCall, BuildInfo buildInfo)
   at LinqToDB.Linq.Builder.MethodCallBuilder.BuildSequence(ExpressionBuilder builder, BuildInfo buildInfo)
   at LinqToDB.Linq.Builder.ExpressionBuilder.BuildSequence(BuildInfo buildInfo)
   at LinqToDB.Linq.Builder.SetOperationBuilder.BuildMethodCall(ExpressionBuilder builder, MethodCallExpression methodCall, BuildInfo buildInfo)
   at LinqToDB.Linq.Builder.MethodCallBuilder.BuildSequence(ExpressionBuilder builder, BuildInfo buildInfo)
   at LinqToDB.Linq.Builder.ExpressionBuilder.BuildSequence(BuildInfo buildInfo)
   at LinqToDB.Linq.Builder.TableBuilder.<>c__DisplayClass11_0.<BuildCteContext>b__1(CteClause cteClause)
   at LinqToDB.Linq.Builder.ExpressionBuilder.BuildCte(Expression cteExpression, Func`2 buildFunc)
   at LinqToDB.Linq.Builder.TableBuilder.BuildCteContext(ExpressionBuilder builder, BuildInfo buildInfo)
   at LinqToDB.Linq.Builder.TableBuilder.BuildSequence(ExpressionBuilder builder, BuildInfo buildInfo)
   at LinqToDB.Linq.Builder.ExpressionBuilder.BuildSequence(BuildInfo buildInfo)
   at LinqToDB.Linq.Builder.SelectManyBuilder.BuildMethodCall(ExpressionBuilder builder, MethodCallExpression methodCall, BuildInfo buildInfo)
   at LinqToDB.Linq.Builder.MethodCallBuilder.BuildSequence(ExpressionBuilder builder, BuildInfo buildInfo)
   at LinqToDB.Linq.Builder.ExpressionBuilder.BuildSequence(BuildInfo buildInfo)
   at LinqToDB.Linq.Builder.ExpressionBuilder.Build[T]()
   at LinqToDB.Linq.Query`1.CreateQuery(ExpressionTreeOptimizationContext optimizationContext, ParametersContext parametersContext, IDataContext dataContext, Expression expr)
   at LinqToDB.Linq.Query`1.GetQuery(IDataContext dataContext, Expression& expr, Boolean& dependsOnParameters)
   at LinqToDB.Linq.ExpressionQuery`1.GetQuery(Expression& expression, Boolean cache, Boolean& dependsOnParameters)
   at LinqToDB.Linq.ExpressionQuery`1.GetForEachAsync(Action`1 action, CancellationToken cancellationToken)
   at LinqToDB.AsyncExtensions.ToListAsync[TSource](IQueryable`1 source, CancellationToken token)

Perhaps I could've changed my explicit string Type discriminator to enum, so EF would be satisfied, but at this point I think that it's worth to wait for the upcoming LinqToDb version 6 and continue my experiments afterwards.