search

ChilliCream / graphql-platform

Welcome to the home of the Hot Chocolate GraphQL server for .NET, the Strawberry Shake GraphQL client for .NET and Banana Cake Pop the awesome Monaco based GraphQL IDE.
https://chillicream.com
MIT License
5.07k stars 723 forks source link

Projection cause "Index out of range" on Database first EF core (net 6) #5229

Open invzz opened 1 year ago

invzz commented 1 year ago

Is there an existing issue for this?

Describe the bug

I Have wrapped an old DB2 database with EF and created a DbContext which does not match the table schema on the database. I had to "split" tables in different entities using the "Owned" properties

so namely the table Viaggi is splitted in infoPax, infoMerci, and so on. for each of the owned type I created a Owned navigation property using EF

for example here entity is of Type Viaggio

 _ = entity.OwnsOne(via => via.InfoPax, entity =>
                {
                    _ = entity.Property(e => e.Capacita).HasColumnName("VIPAT").HasColumnType("smallint(2)").HasDefaultValueSql("0");
                    _ = entity.Property(e => e.Venduto).HasColumnName("VIPAV").HasColumnType("smallint(2)").HasDefaultValueSql("0");
                    _ = entity.Property(e => e.Warning).HasColumnName("VIPAW").HasColumnType("smallint(2)").HasDefaultValueSql("0");
                    _ = entity.Property(e => e.BloccoVendite).IsRequired().HasColumnName("VIPBV").HasColumnType("char(1)").HasDefaultValueSql("' '");
                    _ = entity.Property(e => e.Abilitazione).IsRequired().HasColumnName("VIPFV").HasColumnType("char(1)").HasDefaultValueSql("' '");
                    _ = entity.Property(e => e.TotaleMetriLieari).HasColumnName("VIPML").HasColumnType("decimal(5, 1)").HasDefaultValueSql("0");
                    _ = entity.Property(e => e.ProgressivoBooking).HasColumnName("VIPPB").HasColumnType("integer(4)").HasDefaultValueSql("0");
                }

I Have the usual repository accessing the data and returning an IQueryable<Viaggio>

If I don't use Projections everything works great.

The problem starts when I add [UseProjection] on my query: It all works well until I try to access those owned (in theory non nullable) owned entities which really are in the same table of Viaggi.

That causes

"Index was out of range. Must be non-negative and less than the size of the collection. (Parameter 'index')",

I had a look in the produced Linq query :

 Compiling query expression: 
      'DbSet<Viaggio>()
          .Where(x => x.IdViaggio == __id_0)
          .Select(_s1 => new Viaggio{ 
              IdSocieta = _s1.IdSocieta, 
              InfoPax = _s1.InfoPax != null ? new InfoPax{ BloccoVendite = _s1.InfoPax.BloccoVendite }
               : null,  
                  )
                  .ToList() 
          }
          )'

and it looks fine but :

Steps to reproduce

  1. Using Db First approach create a table on a server and map it onto different entities using ef core table splitting https://docs.microsoft.com/en-us/ef/core/modeling/table-splitting
  2. try to access the nested owned entities using a query in graphql

Relevant log output

"errors": [
    {
      "message": "Unexpected Execution Error",
      "locations": [
        {
          "line": 2,
          "column": 2
        }
      ],
      "path": [
        "viaggiProjection"
      ],
      "extensions": {
        "message": "Index was out of range. Must be non-negative and less than the size of the collection. (Parameter 'index')",
        "stackTrace": "   at Microsoft.EntityFrameworkCore.Query.SqlExpressions.SelectExpression.GetProjection(ProjectionBindingExpression projectionBindingExpression)\n   at Microsoft.EntityFrameworkCore.Query.RelationalShapedQueryCompilingExpressionVisitor.ShaperProcessingExpressionVisitor.VisitExtension(Expression extensionExpression)\n   at System.Linq.Expressions.Expression.Accept(ExpressionVisitor visitor)\n   at System.Linq.Expressions.ExpressionVisitor.Visit(Expression node)\n   at System.Linq.Expressions.ExpressionVisitor.VisitMemberAssignment(MemberAssignment node)\n   at System.Linq.Expressions.ExpressionVisitor.VisitMemberBinding(MemberBinding node)\n   at System.Linq.Expressions.ExpressionVisitor.Visit[T](ReadOnlyCollection`1 nodes, Func`2 elementVisitor)\n   at System.Linq.Expressions.ExpressionVisitor.VisitMemberInit(MemberInitExpression node)\n   at System.Linq.Expressions.MemberInitExpression.Accept(ExpressionVisitor visitor)\n   at Microsoft.EntityFrameworkCore.Query.RelationalShapedQueryCompilingExpressionVisitor.ShaperProcessingExpressionVisitor.ProcessShaper(Expression shaperExpression, RelationalCommandCache& relationalCommandCache, LambdaExpression& relatedDataLoaders, Int32& collectionId)\n   at Microsoft.EntityFrameworkCore.Query.RelationalShapedQueryCompilingExpressionVisitor.VisitShapedQuery(ShapedQueryExpression shapedQueryExpression)\n   at Microsoft.EntityFrameworkCore.Query.ShapedQueryCompilingExpressionVisitor.VisitExtension(Expression extensionExpression)\n   at System.Linq.Expressions.Expression.Accept(ExpressionVisitor visitor)\n   at System.Linq.Expressions.ExpressionVisitor.Visit(Expression node)\n   at Microsoft.EntityFrameworkCore.Query.QueryCompilationContext.CreateQueryExecutor[TResult](Expression query)\n   at Microsoft.EntityFrameworkCore.Query.Internal.CompiledQueryCache.GetOrAddQuery[TResult](Object cacheKey, Func`1 compiler)\n   at Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler.ExecuteAsync[TResult](Expression query, CancellationToken cancellationToken)\n   at Microsoft.EntityFrameworkCore.Query.Internal.EntityQueryProvider.ExecuteAsync[TResult](Expression expression, CancellationToken cancellationToken)\n   at Microsoft.EntityFrameworkCore.Query.Internal.EntityQueryable`1.GetAsyncEnumerator(CancellationToken cancellationToken)\n   at System.Runtime.CompilerServices.ConfiguredCancelableAsyncEnumerable`1.GetAsyncEnumerator()\n   at Microsoft.EntityFrameworkCore.EntityFrameworkQueryableExtensions.ToListAsync[TSource](IQueryable`1 source, CancellationToken cancellationToken)\n   at HotChocolate.Data.ToListMiddleware`1.InvokeAsync(IMiddlewareContext context)\n   at HotChocolate.Types.EntityFrameworkObjectFieldDescriptorExtensions.<>c__DisplayClass2_1`1.<<UseDbContext>b__4>d.MoveNext()\n--- End of stack trace from previous location ---\n   at HotChocolate.Types.EntityFrameworkObjectFieldDescriptorExtensions.<>c__DisplayClass2_1`1.<<UseDbContext>b__4>d.MoveNext()\n--- End of stack trace from previous location ---\n   at HotChocolate.Execution.Processing.Tasks.ResolverTask.ExecuteResolverPipelineAsync(CancellationToken cancellationToken)\n   at HotChocolate.Execution.Processing.Tasks.ResolverTask.TryExecuteAsync(CancellationToken cancellationToken)"
      }
    }
  ]
}

Additional Context?

Im using IBM entity framework core to connect to an AS400 system which has a DB2 instance

Product

Hot Chocolate

Version

12.12.0

PascalSenn commented 1 year ago

@invzz thanks for reporting. We are working on a better ef core integration. At the moment we do not know if EF Core (and what Features of EF Core) is used.

invzz commented 1 year ago

@PascalSenn thanks for your reply, I'm looking forward to it. My workaround is to work with flattened tables without tablesplitting and use automapper later to get the right entity subdivision.

PascalSenn commented 1 year ago

ah you use Automapper? because then this is probably the issue

invzz commented 1 year ago

I'm not using Automapper in graphQL, although I have it registered in services and I use it for other endpoints in a controller which expose some other pieces of functionality (basic crud db interrogations)

I also tried to exclude Automapper ( deleting the .AddAutoMapper(AppDomain.CurrentDomain.GetAssemblies()) line) from the loaded services and excluding it from my controller, but I had the same results


builder.Services

    .Configure<As400Options>(builder.Configuration.GetSection(As400Options.OptionsKey))
    .AddPooledDbContextFactory<AS400Context>(ob =>
    {
        ob.UseDb2(_connectionString, x => x.SetServerInfo(IBMDBServerType.AS400));
        ob.LogTo(Console.WriteLine);
    })
    .AddDbContext<AS400Context>( ob => {
        ob.UseDb2(_connectionString, x => x.SetServerInfo(IBMDBServerType.AS400));
        ob.LogTo(Console.WriteLine);
    })
    .AddTransient<IReadOnlyRepository<ViaggioGviat>, ReadOnlyRepository<AS400Context, ViaggioGviat>>()
    .AddAutoMapper(AppDomain.CurrentDomain.GetAssemblies())
    .ConfigureApi()
    .AddHttpClient()
    .AddGraphQLServer()
    .AddProjections()
    .AddSorting()
    .AddFiltering()
    .RegisterDbContext<AS400Context>()
    .AddQueryType<PaxQuery>();

Another "weird" and related thing is happening :

I changed my context and started using flattened tables which models mirrors the tables in the db schema:

using ConfigurationOptions.Options;
using EF400_Relational.Models.Entities;
using IBM.EntityFrameworkCore;
using Microsoft.EntityFrameworkCore;
using Microsoft.Extensions.Options;

namespace EF400_Relational
{
    public class AS400Context : DbContext
    {   public AS400Context(DbContextOptions<AS400Context> options) : base(options) { }
        public virtual DbSet<TestataGbigt> Testate { get; set; } = null!;
        public virtual DbSet<ViaggioGviat> Viaggi { get; set; } = null!;
        public virtual DbSet<CartaImbarcoGcimb> CarteImbarco { get; set; } = null!;
        public virtual DbSet<NaveTanav> Navi { get; set; } = null!;
        public virtual DbSet<PortoAnpor> Porti { get; set; } = null!;
        public virtual DbSet<MultitrattaGmvia> Multitratte { get; set; } = null!;
        public virtual DbSet<CartaImbarcoOld> CarteImbarcoGassv { get; set; } = null!;

        protected override void OnModelCreating(ModelBuilder modelBuilder)
        {
            ViaggioGviat.BuildModel(modelBuilder, e =>e.ToTable("GVIAT00F", "GNV2008"));
            TestataGbigt.BuildModel(modelBuilder, e =>e.ToTable("GBIGT00F", "GNV2008"));
            CartaImbarcoGcimb.BuildModel(modelBuilder, e =>e.ToTable("GCIMB00F", "GNV2008"));
            NaveTanav.BuildModel(modelBuilder, e =>e.ToTable("TANAV00F", "GNV2008"));
            PortoAnpor.BuildModel(modelBuilder, e =>e.ToTable("ANPOR00F", "GNV2008"));
            MultitrattaGmvia.BuildModel(modelBuilder, e =>e.ToTable("GMVIA00F", "GNVUTL"));
            CartaImbarcoOld.BuildModel(modelBuilder, e =>e.ToTable("GASSV00F", "GNVPROD"));
            CategoriaPaxAncat.BuildModel(modelBuilder, e =>e.ToTable("ANCAT00F", "GNV2008"));
           // _ = modelBuilder.Entity<NaveTanav>(e => {  });
            _ = modelBuilder.Entity<ViaggioGviat>(e =>
            {
                // relazione con testata
                _ = e.HasMany(gviat => gviat.Testate).WithOne(x => x.Viaggio).HasForeignKey(t => new { t.IdSocieta, t.IdViaggio });
                // testate di ritorno
                _ = e.HasMany(gviat => gviat.TestateRitorno).WithOne(x => x.ViaggioRitorno).HasForeignKey(t => new { t.IdSocieta, t.IdViaggioRitorno }).IsRequired();
                // relazione con Nave
                _ = e.HasOne(x => x.Nave).WithMany().HasForeignKey(x => new { x.IdSocieta, x.IdNave }).IsRequired(true);
                // Porto Arrivo
                _ = e.HasOne(x => x.PortoArrivo).WithMany().HasForeignKey(x => x.IdPortoArrivo).IsRequired(true);
                // PortoPartenza
                _ = e.HasOne(x => x.PortoPartenza).WithMany().HasForeignKey(x => x.IdPortoPartenza).IsRequired(true);
                // PortoPrimoScalo
                _ = e.HasOne(x => x.PortoPrimoScalo).WithMany().HasForeignKey(x => x.IdPortoPrimoScalo).IsRequired(false);
            });

            _ = modelBuilder.Entity<MultitrattaGmvia>(e =>
            {
                //viaggio
                _ = e.HasOne(p => p.ViaggioMultitratta).WithOne(p => p.Multitratta).HasForeignKey<MultitrattaGmvia>(p => new { p.IdViaggioMultitratta }).HasPrincipalKey<ViaggioGviat>(x => x.IdViaggio).IsRequired(false);
                //viaggo
                _ = e.HasOne(p => p.ViaggioSingolaTratta).WithOne(p => p.SingolaTratta).HasForeignKey<MultitrattaGmvia>(p => new { p.IdViaggioSingolaTratta }).HasPrincipalKey<ViaggioGviat>(x => x.IdViaggio).IsRequired(false);
            });

            _ = modelBuilder.Entity<TestataGbigt>(e =>
            {
                // relazione con logicalcartaimbarco
                _ = e.HasMany(big => big.CarteImbarcoGcimb).WithOne().HasForeignKey(c => new { c.Societa, c.PrefissoBiglietto, c.IdBiglietto, });
                // relazione con Nave
                _ = e.HasOne(x => x.Nave).WithOne(x => x.Testata).HasForeignKey<TestataGbigt>(x => new { x.IdSocieta, x.IdNave });
                // Porto Arrivo
                _ = e.HasOne(x => x.PortoArrivo).WithMany(p => p.TestateArrivo).HasForeignKey(x => x.IdPortoArrivo).IsRequired(false);
                // PortoPartenza
                _ = e.HasOne(x => x.PortoPartenza).WithMany(p => p.TestatePartenza).HasForeignKey(x => x.IdPortoPartenza).IsRequired(false);
            });

            _ = modelBuilder.Entity<CartaImbarcoGcimb>(e =>
            {
                _ = e.HasOne(x => x.OldCartaImbarcoGassv).WithOne(x => x.NewCartaImbarco).HasForeignKey<CartaImbarcoGcimb>(b => new { b.ViaggioId, b.Societa, b.IdBiglietto, b.ProgressivoCartaImbarco }).IsRequired(false);
                _ = e.HasOne(x => x.CategoriaPax).WithMany().IsRequired(false).HasForeignKey(x => x.IdCategoria);
            });
        }
    }
}

Where the buildModel method is a static method which for every property adds the configuration 

entity.Property(x => x.Myproperty).HasColumnName("ColumnName").HasColumnType("MyType")

In this way I though to avoid using those owned types and table splitting approach.

My query class is a simple one:

namespace EF400RWebApi.QueryTypes
{
    public class PaxQuery
    {
        [UseDbContext(typeof(AS400Context))]
        [UseProjection]
        [UseFiltering]
        public IQueryable<ViaggioGviat> GetViaggiProjection([ScopedService] AS400Context db, int id) => db.Viaggi.Where(v => v.IdViaggio == id).Include(x => x.Nave).Include(x => x.Testate);
 [UseDbContext(typeof(AS400Context))]
        public IQueryable<ViaggioGviat> GetViaggi([ScopedService] AS400Context db, int id) => db.Viaggi.Where(v => v.IdViaggio == id).Include(x => x.Nave);
    }
}
DbSet<ViaggioGviat>()
          .Where(v => v.IdViaggio == __id_0)
          .Include(x => x.Nave)
          .Include(x => x.Testate)
          .Select(_s1 => new ViaggioGviat{ Nave = _s1.Nave != null ? new NaveTanav{ IdNave = _s1.Nave.IdNave }
               : null }
          )

while the 

query projected {
  viaggiProjection(id: 19170) {
    testate {
      idBiglietto
    }
  }
}

works and reply 

 "data": {
    "viaggiProjection": [
      {
        "testate": [
          {
            "idBiglietto": 9482819
          },
          {
            "idBiglietto": 8764575
          },
         ]
      }
] }

everything works if I don't use projections as in GetFullViaggi.

In My use case projections are necessary because the entities have 200+ fields, queries with few fields are much faster (lower in volume: for 1 full viaggio query i get 6 MB of data, if I use projections i get few kb)

for clarity I here the navigation properties inside viaggioGviat

class ViaggioGviat {
         // omitted for brevity.... 
         // reference to Nave
        public string? IdNave { get; set; }
        public virtual NaveTanav Nave { get; set; } = null!;

        // reference to Porto
        public string? IdPortoPartenza { get; set; }
        public virtual PortoAnpor PortoPartenza { get; set; } = null!;

        // reference to porto
        public string? IdPortoArrivo { get; set; }
        public virtual PortoAnpor PortoArrivo { get; set; } = null!;

        // reference to porto
        public string? IdPortoPrimoScalo { get; set; }
        public virtual PortoAnpor PortoPrimoScalo { get; set; } = null!;

        //reference  to testata
        public virtual ICollection<TestataGbigt>? Testate { get; set; }
        public virtual ICollection<TestataGbigt>? TestateRitorno { get; set; }

        // references Multitratta
        public virtual MultitrattaGmvia Multitratta { get; set; } = null!;
        public virtual MultitrattaGmvia SingolaTratta { get; set; } = null!;

 // omitted for brevity.... 
}

I'm now trying to use the github repo instead of the nuget packages so I can further investigate through brakepoints what's happening, Hotchocolate project is huge and I'm not currently been able to set it up as I get some reference errors on builds

invzz commented 1 year ago

@PascalSenn

looks like my problem is here 

 public static Expression NotNull(Expression expression)
    {
        return Expression.NotEqual(expression, _null);
    }

    public static Expression NotNullAndAlso(Expression property, Expression condition)
    {
        return Expression.Condition(
            NotNull(property),
            condition,
            Expression.Default(property.Type));
    }

Looks like theNotNullAndAlso is adding the condition which causes my problems: