dotnet / efcore

EF Core is a modern object-database mapper for .NET. It supports LINQ queries, change tracking, updates, and schema migrations.
https://docs.microsoft.com/ef/
MIT License
13.76k stars 3.18k forks source link

Investigate Hot Reload issues with EF Core #26914

Open panthernet opened 2 years ago

panthernet commented 2 years ago

I've ported my project from v5 to v6 EF Core. Previously everything worked fine but now I'm getting random translation exceptions from EF.Functions.Like calls. I monitor the same operation and see the same data coming from input but sometimes it passes fine and sometimes throws an exception.

Behavior looks random, I can't understand what's causing it. But I see it is a cascade failure. Once one error pops, all queries with LIKE function starts to fail on execution: different tables, not a single one.

Call example:

public static async Task SetCacheLastAccess(string id, string type)
{
   await using var db = new DbContext();
   var entry = await db.Cache.FirstOrDefaultAsync(a => EF.Functions.Like(a.Id, id) && EF.Functions.Like(a.Type, type));
}

...
       public string Type { get; set; } = "-";
        public string Id { get; set; }
...

Database

            modelBuilder.Entity<ThdCacheEntry>().HasIndex(u => u.Id);

            modelBuilder.Entity<ThdCacheEntry>().ToTable("cache");

            modelBuilder.Entity<ThdCacheEntry>().Property(a => a.Id).HasColumnName("id");
            modelBuilder.Entity<ThdCacheEntry>().Property(a => a.Type).HasColumnName("type");
            modelBuilder.Entity<ThdCacheEntry>().Property(a => a.LastAccess).HasColumnName("lastAccess");
            modelBuilder.Entity<ThdCacheEntry>().Property(a => a.LastUpdate).HasColumnName("lastUpdate");
            modelBuilder.Entity<ThdCacheEntry>().Property(a => a.Content).HasColumnName("text");
            modelBuilder.Entity<ThdCacheEntry>().Property(a => a.Days).HasColumnName("days");

image

Exception details

The LINQ expression 'DbSet<ThdCacheEntry>()
    .Where(t => __Functions_0
        .Like(
            matchExpression: t.Id, 
            pattern: __id_1) && __Functions_0
        .Like(
            matchExpression: t.Type, 
            pattern: __type_2))' could not be translated. Additional information: Translation of method 'Microsoft.EntityFrameworkCore.DbFunctionsExtensions.Like' failed. If this method can be mapped to your custom function, see https://go.microsoft.com/fwlink/?linkid=2132413 for more information.
Translation of method 'Microsoft.EntityFrameworkCore.DbFunctionsExtensions.Like' failed. If this method can be mapped to your custom function, see https://go.microsoft.com/fwlink/?linkid=2132413 for more information. Either rewrite the query in a form that can be translated, or switch to client evaluation explicitly by inserting a call to 'AsEnumerable', 'AsAsyncEnumerable', 'ToList', or 'ToListAsync'. See https://go.microsoft.com/fwlink/?linkid=2101038 for more information.
   at Microsoft.EntityFrameworkCore.Query.QueryableMethodTranslatingExpressionVisitor.<VisitMethodCall>g__CheckTranslated|15_0(ShapedQueryExpression translated, <>c__DisplayClass15_0& )
   at Microsoft.EntityFrameworkCore.Query.QueryableMethodTranslatingExpressionVisitor.VisitMethodCall(MethodCallExpression methodCallExpression)
   at Microsoft.EntityFrameworkCore.Query.QueryableMethodTranslatingExpressionVisitor.VisitMethodCall(MethodCallExpression methodCallExpression)
   at Microsoft.EntityFrameworkCore.Query.QueryCompilationContext.CreateQueryExecutor[TResult](Expression query)
   at Microsoft.EntityFrameworkCore.Storage.Database.CompileQuery[TResult](Expression query, Boolean async)
   at Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler.CompileQueryCore[TResult](IDatabase database, Expression query, IModel model, Boolean async)
   at Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler.<>c__DisplayClass12_0`1.<ExecuteAsync>b__0()
   at Microsoft.EntityFrameworkCore.Query.Internal.CompiledQueryCache.GetOrAddQuery[TResult](Object cacheKey, Func`1 compiler)
   at Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler.ExecuteAsync[TResult](Expression query, CancellationToken cancellationToken)
   at Microsoft.EntityFrameworkCore.Query.Internal.EntityQueryProvider.ExecuteAsync[TResult](Expression expression, CancellationToken cancellationToken)
   at Microsoft.EntityFrameworkCore.EntityFrameworkQueryableExtensions.ExecuteAsync[TSource,TResult](MethodInfo operatorMethodInfo, IQueryable`1 source, Expression expression, CancellationToken cancellationToken)
   at Microsoft.EntityFrameworkCore.EntityFrameworkQueryableExtensions.ExecuteAsync[TSource,TResult](MethodInfo operatorMethodInfo, IQueryable`1 source, LambdaExpression expression, CancellationToken cancellationToken)
   at Microsoft.EntityFrameworkCore.EntityFrameworkQueryableExtensions.FirstOrDefaultAsync[TSource](IQueryable`1 source, Expression`1 predicate, CancellationToken cancellationToken)
   at ThunderED.DbHelper.<SetCacheLastAccess>d__35.MoveNext() in 

Include provider and version information

EF Core version: 6.0.0 Database provider: Microsoft.EntityFrameworkCore.Sqlite 6.0.0 Target framework: .NET 6.0 Operating system: WIn10 x64 21H1 IDE: Visual Studio 2022 17.0.2 Project: Blazor Server-Side

panthernet commented 2 years ago

UPDATE: Okay not sure by 100% but looks like it happens after I use HOT RELOAD button in VS 2022 to apply changes OR reload happens automatically on code change. I use server-side Blazor for the project.

UPDATE: Confirmed. HOT RELOAD is the bad guy.

HOT RELOAD logs

THDWebServer (Web server): Hot reload session complete
THDWebServer (Web server): Hot reload session started
THDWebServer (Web server): Sending updates to running application
THDWebServer (Web server): Updates applied successfully
ajcvickers commented 2 years ago

@panthernet Specifically, what changes are you applying with hot reload? For example, are you changing the EF model, the query, something else related to EF, or code not specific to EF at all? It would be great if you could attach a small project that results in this behavior with instructions of what you change under hot reload to make it happen.

panthernet commented 2 years ago

@panthernet Specifically, what changes are you applying with hot reload? For example, are you changing the EF model, the query, something else related to EF, or code not specific to EF at all? It would be great if you could attach a small project that results in this behavior with instructions of what you change under hot reload to make it happen.

It happens on every hot reload apply. The changes not related to database. I don't have possibility to create the test project right now. On vacation 🙂

Tim-Andersen commented 2 years ago

I'm also running into this in a larger web app, but looks pretty easy to reproduce, see attached zipped-up VS Solution...

HotReloadEFCoreLikeIssue (2).zip

...with the main code being in Index.cshtml.cs:

    public void OnGet()
    { 
        People = _context.People
            .Where(p => EF.Functions.Like(p.EmailAddress, "%@outlook.com"))
            .ToList();

        // Uncomment below line and save.
        //var abc = "def";
    }

As prompted above, just uncomment the above line and it should throw the following exception:

System.InvalidOperationException: 'The LINQ expression 'DbSet() .Where(p => __Functions_0 .Like( matchExpression: p.EmailAddress, pattern: "%@outlook.com"))' could not be translated. Additional information: Translation of method 'Microsoft.EntityFrameworkCore.DbFunctionsExtensions.Like' failed. If this method can be mapped to your custom function, see https://go.microsoft.com/fwlink/?linkid=2132413 for more information. Either rewrite the query in a form that can be translated, or switch to client evaluation explicitly by inserting a call to 'AsEnumerable', 'AsAsyncEnumerable', 'ToList', or 'ToListAsync'. See https://go.microsoft.com/fwlink/?linkid=2101038 for more information.'

(.NET 6's new web app project template without a Startup class threw me for a loop! And it's been a while since I setup a new web app from scratch, so hopefully I set it up correctly)

maknapp commented 2 years ago

I am using efcore 3 + postgresql with npgsql that supports translating the Regex.IsMatch function. Every time a hot reload happens, the query that uses Regex.IsMatch fails with the same "could not be translated" error. This appears to be any change - adding a character to a string in the same controller, different controller, different unrelated class in a second referenced project all causes the query to fail.

ajcvickers commented 2 years ago

Note from triage: putting this in 7.0 to investigate the experience of EF Core with hot reload.

Liero commented 2 years ago

+1 I have this problem too. I'm using SQL Server and Blazor Server Side and dotnet watch cli

@ajcvickers: Could this be related to Roslyn, or .NET tooling? In that case, at that time you investigate it, won't it be too late for the the other teams responsible for fixing it? Thanks!

roji commented 2 years ago

https://github.com/npgsql/efcore.pg/issues/2231 sounds suspiciously close to this as well...

midgleyc commented 2 years ago

When running hot reload with lazy loading (and nullable reference types, if that's relevant), I sometimes get the error:

System.NotImplementedException: This is a DynamicProxy2 error: The interceptor attempted to 'Proceed' for method 'Microsoft.EntityFrameworkCore.Infrastructure.ILazyLoader get_LazyLoader()' which has no target. When calling method without target there is no implementation to 'proceed' to and it is the responsibility of the interceptor to mimic the implementation (set return value, out arguments etc)
         at Castle.DynamicProxy.AbstractInvocation.ThrowOnNoTarget()
         at Castle.DynamicProxy.Internal.CompositionInvocation.EnsureValidTarget()
         at Castle.Proxies.Invocations.IProxyLazyLoader_get_LazyLoader.InvokeMethodOnTarget()
         at Castle.DynamicProxy.AbstractInvocation.Proceed()
         at Microsoft.EntityFrameworkCore.Proxies.Internal.LazyLoadingInterceptor.Intercept(IInvocation invocation)
         at Castle.DynamicProxy.AbstractInvocation.Proceed()
         at Castle.Proxies.DraftJobProxy.get_LazyLoader()
         at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.InternalEntityEntry.ReadPropertyValue(IPropertyBase propertyBase)
         at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.InternalEntityEntry.get_Item(IPropertyBase propertyBase)
         at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.InternalEntityEntry.SetIsLoaded(INavigationBase navigation, Boolean loaded)
         at Microsoft.EntityFrameworkCore.Query.QueryContext.SetNavigationIsLoaded(Object entity, INavigationBase navigation)
         at lambda_method768(Closure , QueryContext , DbDataReader , ResultContext , SplitQueryResultCoordinator )
         at Microsoft.EntityFrameworkCore.Query.Internal.SplitQueryingEnumerable`1.AsyncEnumerator.MoveNextAsync()

Once it happens, it happens consistently, but it happens rarely.

PudottaPommin commented 2 years ago

I can confirm happening this with Like on SQL Server and ILike for Npgsql. Both happen when app is hot reloaded. Makes no difference if change was in C# or template ( cshtml ).

// fails on this query after hot reload
var query = context.Products.Where(p => EF.Functions.ILike(p.ProductCode, productCode)).ToList();

// works on this one
var query = context.Products.Where(p => p.ProductCode.ToLower().Equals(productCode)).ToList();

Also, I noticed it happen when using .OrderBy(_ => EF.Functions.Random()). Changing to .OrderBy(_ => Guid.NewGuid()) works without hitch.

mhosman commented 2 years ago

This happened to me today too. I had to restart the app and it worked.

changhuixu commented 2 years ago

It happened to me too. in .NET 6 and EF Core v6.0.7 I am using dotnet watch run. The error comes out randomly upon hot reload, even though the changes are not in the query. I have to restart the app to make it work.

rudydalsantomed commented 2 years ago

Is there any news for this issue? I experienced this, always after hot relaod, if using EF.Function.Like. Restarting the app solve the problem. (.NET 6.0.8, EF 6.0.8)

gasksoft commented 2 years ago

Is there any news for this issue? I experienced this, always after hot relaod, if using EF.Function.Like. Restarting the app solve the problem. (.NET 6.0.8, EF 6.0.8)

I have the same problem, on Blazor Server, restarting the app obviously fixes it, but it's annoying to restart every now and then to see any minor changes. The funny thing is that this code is in a component, and the queries that are executed within it fail, however in the parent component, the queries are executed normally

BlueManiac commented 2 years ago

I'm hitting this issue as well but using EF.Functions.DateDiffDay. Seems to trigger nearly always after a hot reload.

audacity76 commented 1 year ago

Same here. Any news regarding this issue?

nathanvj commented 1 year ago

Same here. Using the EF.Functions.Random() function breaks the query after hot reload. Restarting the application fixes the issue.

rshekhtm commented 1 year ago

Same issue with hot reload and EF.Functions.ILike usage. The hacky workaround I put in place was to catch the InvalidOperationException and invoke a query without ILike that is similar enough for development purposes, but is not suitable for production.

wanghucheng66 commented 1 year ago

Got same issue with hot reload and lazy query "Method 'get_LazyLoader' on type 'Castle.Proxies.EventLogProxy' from assembly 'DynamicProxyGenAssembly2, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null' is overriding a method that has been overridden." net 6.0 efcore 7.0.5

luccawilli commented 10 months ago

I still got the issue, VS 2022 17.8.3, Npgsql.EntityFrameworkCore.PostgreSQL 6.0.6

vallgrenerik commented 9 months ago

I think I'm also hitting this error, using Blazor WASM with sqlite.

asmolarczyk commented 5 months ago

I also have this bug when using hot reload in a MAUI app with EF Core 8.0.4 (but I also tried various other EF Core version down to 5.0.17). I attached a very minicmalistic sample app demonstrating the problem. In this app the problem occurs always after a hot reload.

Of course the suggested workaround 'switch to client evaluation explicitly by inserting a call to ... ToList()' does work, but I'm also using Include(...) to eagerly load navigation properties and this does not work with ToList(). Unfortunately Hot Reload is completely unusable for me and debugging/developing with Maui is quite a pain.

Any help is really appreciated!

https://github.com/dotnet/efcore/assets/77154858/4adb9205-be37-46e0-b6f4-072813904ab5

MauiApp4.zip

vallgrenerik commented 5 months ago

I also have this bug when using hot reload in a MAUI app with EF Core 8.0.4 (but I also tried various other EF Core version down to 5.0.17). I attached a very minicmalistic sample app demonstrating the problem. In this app the problem occurs always after a hot reload.

Of course the suggested workaround 'switch to client evaluation explicitly by inserting a call to ... ToList()' does work, but I'm also using Include(...) to eagerly load navigation properties and this does not work with ToList(). Unfortunately Hot Reload is completely unusable for me and debugging/developing with Maui is quite a pain.

Any help is really appreciated!

devenv_NRNYiCt721.mp4 MauiApp4.zip

This is exactly what I'm experiencing! I love to use both ef core and hot reload and it's so sad that this is what stopping from getting quick feedback from changes.

@ajcvickers do you think that is something that's easy to fix or is this just a edge case? 😊

ajcvickers commented 5 months ago

@vallgrenerik I think if we were going to work on this, then it would mostly about providing better messages in cases where it doesn't work, such as changing a query.

asmolarczyk commented 5 months ago

Have you watched my Video? I'm using Count() on a DbSet and I have not changed this really simple query. I only changed a ContentPage. I think a better message would not help here. Unfortunately EF Core + Hot Reload is completely broken on Maui. I wonder whether this is something that can be fixed by you or by the Hot Reload/.Net guys? Btw: I also tried Net9 preview + EF Core 9. Problem persists...