Closed AndiRudi closed 1 year ago
@PeterLindgrenNorrsken I find it better specifying it in the query itself rather than somewhere in the configurations.
@gdoron we agree on that, that's what I meant with "on a per-query basis", maybe I should have expressed that differently.
It would be nice to at least be able to eagerly load instead of explicitly configuring the include/theninclude for the whole tree which can become quite cumbersome.
For children navigration properties I've managed to get it working via expression trees recursively
private static bool IsSameKey<T>(this DbContext db,T lhs, T rhs) where T : class
{
var lhsKeys = db.Entry(lhs).Metadata.FindPrimaryKey().Properties.Select(x => x.GetGetter().GetClrValue(lhs));
var rhsKeys = db.Entry(rhs).Metadata.FindPrimaryKey().Properties.Select(x => x.GetGetter().GetClrValue(rhs));
return lhsKeys.SequenceEqual(rhsKeys);
}
public static void AddChildren<T>(this DbContext db, T entity) where T : class
{
var query = db.Set<T>().Where(x => db.IsSameKey(x, entity));
foreach (var navProperty in db.Entry(entity).Metadata.GetNavigations())
{
var pe = Expression.Parameter(typeof(T));
var me = Expression.Property(pe, navProperty.Name);
query.Include(Expression.Lambda<Func<T, object>>(me, pe)).SingleOrDefault();
if (!navProperty.IsCollection())
typeof(DbExtensions).GetMethod(nameof(AddChildren))
.MakeGenericMethod(navProperty.GetTargetType().ClrType)
.Invoke(null, new [] { db, navProperty.GetGetter().GetClrValue(entity) });
}
}
@rowanmiller Is there any new status about this? Because we have a huge application that uses lazy loading and if there won't be lazy loading in the future we have to start refactoring our approach and bring in the context in every method (which I don't like but we have to move to .net core)
@AndiRudi EF6 is working pretty well with Core as far as I can tell. I haven't had any problems with it. I'm pretty sure the official answer for lazy loading with EF7 is "not RTM" but I don't have a cite for you right now.
@tuespetre Not high on the list: https://github.com/aspnet/EntityFramework/wiki/Roadmap
@ErikEJ I honestly do not understand what "lazy loading" isn't an anti pattern. You don't control the query, and you might hit N+1 queries to the DB without even knowing. As it's now, if you try to fetch an entity without fetching it with a inner/outer join, you will hit an exception/empty set (depends if you init the collecetion in the ctor).
I actually like that this "feature" was left behind for EF Core, and I don't miss it.
@gdoron I agree 100%
@ErikEJ @gdoron just like any pattern, its "anti-ness" is context-dependent
EF 6 to EF 7/EF Core 1 is not unlike moving from ASP.NET Web Forms to ASP.NET MVC. You are going from something that is more robust and mature to something primitive. Personally, I'm far more worried about ASP.NET. Leaving out lazy loading the long run seems unacceptable. People will keep asking for it. It is on the road map as a high priority item, just not a critical one. Also, I think change tracking using proxies will be needed to get decent performance. If you are working with a lot of entities, performance is abysmal in EF 6 if you aren't using proxies. It is much worse than the original EF that wasn't using POCOs. I have a feeling Microsoft wants to not use proxies and that is why they aren't prioritizing lazy loading. I think they are trying to keep things as simple as possible so as to not confuse people. I think they are worried that proxies complicate things with regard to debugging. I was told this with regard to EF 6 a while ago.
Well, if they don't implement lazy loading, I hope they really do improve the way Include()s are handled so that it doesn't create a giant multiply. I hope it works for projections as well.
@tuespetre @jemiller0
I don't think "lazy loading" in the context used here is a good design at all. It mix POCOs with implicit database querying. The context should be the querying facade, not a POCO.
Except from the design that you can argue about, having proxies implicitly querying the DB is a very risky feature that I honestly do not understand the need for that. If you need to a navigation property explicitly query it from the DB and via the navigation property itself. This way you'll never get bitten by an N+1 on production. See how RavenDb prevents you from doing that.
And regarding the hope that lazy loading will come in a latter stage, I honestly don't think it will happen if RTM will be without it.
In order to use proxies you must mark all the navigation properties and collections as virtual, if it will not be required in V1 but will be in V2, too much code will break.
In some cases, I think N+1 can actually be faster than doing a massive join. If you don't want the feature, disable it. I think you can already do this in EF 6. So, I don't know why people are complaining.
After some reflection and trialling with EF Core, I've come to the conclusion that the lack of lazy loading makes it unusable for us. Our ORM coding style (first with NHibernate and later with EF4/5/6) is to use lazy loading and write code which freely and naively navigates foreign key properties and collections. We then do profiling to identify N+1 select issues which have a performance impact and eliminate them by adding appropriate eager or explicit loading, with comments to explain the exceptional circumstances which required it.
If you have to use eager loading, this means you have to scatter extra loading logic through code which should be focused on the business domain. Having to write separate code for data loading creates a maintenance burden to keep this loading code in sync with what the business logic requires. It will substantially increase the risk of regressions where necessary data has not been loaded in advance, or the risk of leaving in code which eagerly loads data which is no longer actually required. In large applications where the business logic is complex and there are several developers, this is not practical.
The situation is especially bad where you have shared business functions which are called from multiple entry points. In such cases there is a high risk of regressions where the function works from some entry points but not others because of assumptions it makes about what data has already been loaded, and these are very likely to be missed in regular testing.
I understand the performance implications of lazy loading, but having clean and maintainable business code is our primary concern, and that means relying on lazy loading for the 90% or more of our code where there is no significant performance gain to be obtained through eager loading. For us, not having lazy loading misses half the reason we use an ORM framework, and we will stick to EF6 for now rather than abandoning our current code style.
+1 Lazy Loading. We are in the same position as john-surcome. We use Lazy Loading unless our profiling states its a problem; yes it's an issue for inexperienced ORM developers who hit the SELECT N+1 problem but then that's why we have lead devs who know what they are doing.
To say "it causes issues for devs so we shouldn't have it at all" is, IMO, a poor reason for adding additional complexity to every scenario to force either Include or additional queries on the developer. If they know enough to add the additional queries, they know enough to understand how ORMs can be susceptible to SELECT N+1
You wouldn't try to fix a car engine without understanding the basics of mechanics. Why should using ORMs be any different?
We use NHibernate, without any performance issues whatsoever, but want to move to EF7 and .NET Core (as it looks like NH will never be .NET Core compatible) but unless there is a modicum of feature compatibility then it's a non-starter for shops like us.
@Garrus007 Include() and ThenInclude() are extension methods for IQueryable
defined in the Microsoft.Data.Entity namespace. Are you missing a using statement with that namespace in your code?
@divega this helped me figure out why it wasn't coming up for me. Wasn't aware it was only for IQueryable.. I got my Include method working now, however I can't seem to do a .ThenInclude it doesn't give me any properties only methods inside the lamba.
Would I use ThenInclude to get another set of navigation properties related to the navigation property I've just included?
@ElvisHalil From what you are describing I suspect you might be hitting a known issue in the Intellisense functionality included in VS: https://github.com/dotnet/roslyn/issues/8237. The consequence of that limitation is that for certain scenarios Intellisense won't show the full set of properties you could use in the lambda, but if you just finish typing the lambda with the name of the property you wanted to include it should compile and work just fine. E.g.:
var query = context.Orders.Include(o => o.Lines).ThenInclude(l => l.Product);
@divega Thank you sir, this indeed does still work for me, just a Intellisense problem.
When doing a chain of Includes and ThenIncludes to be returned by a Web API I only get a single record back with only the first relationship from the chain in the HTTP GET response. When I remove the whole chain and only leave the GetAll() method I get all the records back but without any relationships because I don't specify them any longer. I'd like to be able to get all the relationships back with the Web API if possible.
Is this a known issue for MVC 6 Web API and EF core?
@ElvisHalil Could you please start a separate issue and provide the relevant snippets of code?
@divega new issue posted: https://github.com/aspnet/EntityFramework/issues/5357
Is there any date when the feature will be available? These .Include statements I have to write are ridiculous, it's causing the application to be slower and/or ruining my application logic that was written with lazy loading in mind and I would rather wait for the feature than rework the application.
When using in memory database for testing I'm getting an "auto" eager loading.
@mariustom actually, no. If you are using the ORM as intended it will make your application much faster by reducing number of calls to the DB.
@gdoron Speed is not the most important thing. For a good application design its often acceptable to have less speed in the first place, but a more readable solution in the long run. I also figured that sometimes lazy loading ist even faster if done right. Above all if you want to use EF like ActiveRecord in Rails Lazy Loading is a must have!
@AndiRudi I still believe(and have experience that proves it) that the feature that people here call "lazy loading" is an anti pattern. Amature developers using it wrong without being aware of the performance problem it can cause. Experienced developers don't need it as they can design their application better.
I for instance I would love not to have the feature or at least being able to disable it.
But I feel like I'm the minority, well at least in this thread...
@gdoron It's about creating an application, that is easy to understand. In my case its because I want to write IQueryable Extensions that can do something like customer.Orders.Active() which return all the active orders. There is no perfomance bottleneck in that call (because paging comes later) but you need lazy loading to do this because in an IQueryable Extension you don't have access to the context.
@gdoron Lazy loading can demonstrably reduce the amount of code you have to write, reduce the chance that the code you do write contains bugs, and it can also give performance improvements because in some circumstances you can avoid inadvertently loading the same data multiple times. These are all good practice. With general good practice, an experienced developer can leverage all these benefits throughout their code and rarely hit performance issues.
I don't really see how eager or explicit loading code can be a better coding pattern than the lazy equivalent, which is usually no code at all, except in those cases where the eager/explicit code is significantly faster.
@gdoron I feel this is akin to the "" vs "m" vs "nothing at all" debate for class level naming conventions. There is no right and wrong
I still believe (and have experience that proves it) that having an ORM without Lazy Loading is a sop to inexperienced or poor developers. Not understanding how ORMS work and checking your SQL statements in profiler is a classic ORM mistake.
In my case, inexperienced developers have built applications that loaded the whole database at once. They had turned Lazy Loading off everywhere because it didn't suit them and it was easier then writing Include/Fetches everywhere.
The fix? Optimal use of Includes/Fetches where appropriate and Lazy Load the rest. Both applications are now running sweetly with minimal load on the database
The motto is, give developers the choice. You don't want it, fine - allow disabling either globally or on a relationship by relationship level (aka NHibernate) since you know how to achieve your goal otherwise, but to not have the option at all is a bad call IMO.
And don't feel we're picking on you :) we just really want to make sure that for us, EF gets this now. Otherwise it probably never will, and we are likely to be stuffed in the .NET Core space.
Lazy loading is listed as a “critical feature” not in 1.0 https://blogs.msdn.microsoft.com/dotnet/2016/05/16/announcing-entity-framework-core-rc2/, not a high priority or low priority one, a critical one. So, it would appear that it’s going to get in there eventually. It’s just that the EF team had other things that it needed to implement first. Yes, developers that don’t know what they are doing can get burned by lazy loading which can cause horrible performance. At the same time, I think it can actually improve performance in some cases (although admittedly, you are probably more likely to get burned by it), at least from what I’ve seen in some of the applications that I’ve written using other ORMs such as JPA/EclipseLink. I.e. with caching I think it can be faster than doing a big JOIN multiply. EF 7/EF Core 1 is supposed to have a different way of getting the data without having to do massive JOINs. So, maybe it’s less of an issue with EF Core 1 doing a lot of Includes(). I’m not sure I would be opposed if it were turned off by default. Basically, it should be as simple as setting a property or calling a method to globally turn it on or off. Also, I think that ability would help out a lot with regard to serializing data (for example returning data over a web service) retrieved through the ORM. I can see why maybe the EF team would want to be able to avoid using proxies. I don’t know if that was part of their motivation when they were thinking of not implementing it. However, I think you need proxies to get decent performance. I.e. you need it because you need event driven change tracking. Snapshot uses too much memory and is slow, making it much less usable if you have a lot of entities.
Jon
From: Graham Bunce [mailto:notifications@github.com] Sent: Friday, May 27, 2016 9:18 AM To: aspnet/EntityFramework EntityFramework@noreply.github.com Cc: Jon Miller jemiller@uchicago.edu; Mention mention@noreply.github.com Subject: Re: [aspnet/EntityFramework] Lazy Loading (#3797)
@gdoronhttps://github.com/gdoron I feel this is akin to the "" vs "m" vs "nothing at all" debate for class level naming conventions. There is no right and wrong
I still believe (and have experience that proves it) that having an ORM without Lazy Loading is a sop to inexperienced or poor developers. Not understanding how ORMS work and checking your SQL statements in profiler is a classic ORM mistake.
In my case, inexperienced developers have built applications that loaded the whole database at once. They had turned Lazy Loading off everywhere because it didn't suit them and it was easier then writing Include/Fetches everywhere.
The fix? Optimal use of Includes/Fetches where appropriate and Lazy Load the rest. Both applications are now running sweetly with minimal load on the database
The motto is, give developers the choice. You don't want it, fine - allow disabling either globally or on a relationship by relationship level (aka NHibernate) since you know how to achieve your goal otherwise, but to not have the option at all is a bad call IMO.
And don't feel we're picking on you :) we just really want to make sure that for us, EF gets this now. Otherwise it probably never will, and we are likely to be stuffed in the .NET Core space.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/aspnet/EntityFramework/issues/3797#issuecomment-222158011, or mute the threadhttps://github.com/notifications/unsubscribe/APMZ5Vc2VcGJ2Oy96Drw5DpDJz_9lGjbks5qFv0EgaJpZM4GlnTm.
My vote for lazy loading. It's very simplify developer's life. At this moment I'm trying to do simple Lazy loading functionality for SQLite,because EF7 can't do it But lazy loading also requires Change tracker. Therefore, the EF will no longer be needed for me at all,simple SQLite ORM will be enough
When using in memory database for testing I'm getting an "auto" eager loading.
@lucabriguglia that sounds unexpected and could be a bug instead of a feature. If you have a repro, could you create an issue with it?
just my 2 cents for lazy/eager loading. It would have been better if they weren't included at all, but if they are they should be disabled by default.
@gdoron With properly configured PostgreSQL pooling and caching, lazy loading is faster, regardless of number of calls (since the pooler responds to them, not the database). It might seem that caching the whole "big" query would be even better, but no, that would cause the pooler to cache big results of multiple slighly different result sets and run out of memory. Instead, it can cache more smaller queries and quickly send results when needed.
without lazy/eager loading needs many change in previous project for migrate to Entity Framework Core, please add lazy/eager loading to version 1.0 milestone.
+1 Lazy Loading. ^^
I'm getting into serious trouble. I'd like to retain our backend code base in .net, but our developers uses macs because we do a lot in web development on the frontend. That means they can't develop for the backend because we cannot use .net core with missing lazy loading and we cannot develop on mac without .net core. Besides ... all our developers think that a big business application is best to be written with lazy loading. When I read the current roadmap it seems that lazy loading will take a long time to be there. Now we have more and more people talking about a shift to rails. Is there any way to speed up the decission and to make this work until end of the year?
@AndiRudi isn't setting up the project(s) to run on mono an options since you're aiming for .NET Core anyway? As far as I know EF6 can run pretty well on mono.
@AndiRudi to be honest, if you're familiar with rails and use mac, I would simply use rails... AFAIK Visual Studio isn't supported on mac nor there are plans for doing that . Visual studio code (which is cross platform) is a good idea, but from my XP, it's really nothing compared to VS. Did you try working with it? Maybe it was because I used it a while back, I dunno.
That aside, lazy load feature of an ORM is seriously the last reasonable reason to leave .NET for.
@gdoron Visual Studio Code is fantastic but feature wise it's still quite a mile from Visual Studio itself. Also JetBrains is developing Rider to be a truly cross-platform C# IDE and it's progressing really nice and as it stands I will probably move my C# development to it once it's more stable; both on mac and windows.
And yeah, the lack of lazy load is a poor excuse to rewrite your entire codebase. At the very least see if mono is viable as a short term solution until EF7 team gets to implementing lazy loading if you cannot wait for it.
@AndiRudi and BTW, regarding your snippet code miles up...
Course course = context.Courses.SingleOrDefault(c => c.Id = 1);
course.Cancel();
I think your real issue here is that you don't have POCO, not lazy loading.
Your entities should not have any methods on them (not even ToString
IMHO)
When you mix your DAL with your BL like that, as you see, there're troubles.
@ptMuta Thats for the tip with mono I didn't know that.
@gdoron: That is exactly my issue. I actually want to do mix that. We've played with different implementations in the beginning of the project but the mix is the one that works the best. We found it has the benefit of not writing to much code, is easy to test. similiar to active record in rails, leverages the unit of work and it's easy to write a complex business logic layer. I fear that the easy way is not gonna be supported. If you / anyone has an alternative approach I'd really like to see that... because I really don't want to leave .NET
because I really don't want to leave .NET
Steve Balmer is crying :smile:
Anyway, it's hard to explain in couple of lines why it's wrong, but there're good reasons why people in .NET are using POCOs and people in Java are using POJOs. It's good for tests (no idea why you think the other way around), it's good for portability and it's good for changes and enhancement.
If you're not already, I suggest you take a look on AutoMapper, it will probably solve many of the issues you have with the separation.
@gdoron If you follow Domain Driven Desing principles, I don't understand why you can't use Entities/Aggregate and ValueObject to send and retrieve information from database. In our own big projects, we develop a system of Attributes that defines in one hand Business logic (for exemple Required or MaxLength), and can, on the other hand, be used to define Database constraints. This also allow use to avoid to use dto, especially in the case when a single object target multiples tables OR one table target mutliple objects. In our architecture for exemple , we have a master entity that contains a value object, both this objects are stored in the same table. I don't really see how we can do this in a comprehensive and clear way with DTOs or something mechanism like that. So, in this case, we use EF to map property based on attribute definition and architecture definition. On the other side, we have created our own LazyLoading, by creating a private member for the object to lazy load, and in the public property, in get method, we check if private member is null, if yes, we call or own .Load() extension method with the GUID of the master object. This works pretty well for our needs. Also, we se the default behavior for mapping business object to Ignore for properties (the only that are mapped are the one with or own attributes on top of them), and if I'm not wrong, having one or many methods/members/other C# objects should not have impact on EF Core work.
In my view Lazy Loading was perhaps the essential feature for EntityFramework. Not having it in EF Core 1 it would have been better to not call it Entity Framework at all.
Now, apart from the discussion of what a developer should or should not do, I personally think the developer should decide this for himself and his specific scenario. Orientation on patterns is a good idea, as it is a common ground, however, devotion to patterns, you may as well stop using your brain...
As I said, one of the absolute main reasons for me to use EF at all, was the Lazy Loading feature as it reduces the effort to ad hoc query a database significantly. Having now to decide during development what I need to load explicitly is creating a notable extra on my work. And reduces my agility. Which is a setback, now that we are all supposed to be agile.
And let's be serious, on the performance debate, anyone doing really serious db related work hasn't and won't use EF on mission critical performance heavy operations. Those ones in my view even today will revert to what their DBMS offers for partitioning data, using stored procedures, queries optimized by execution plans which are taking in consideration other relevant operations on the db out of the application bounds etc., and eventually use EF on a top of a data layer, where performance is not that great an issue (I refer to what EF does, when it creates joins, how a db is designed etc.) I at least can't imagine any other user.
Without Lazy Loading, the only unique selling proposition of EF Core is, that you have one API over many DBMS. That is the only reason right now, why I am sticking to it. Besides of all the efforts I invested already in porting from EF6.
And while I am in rage mode, one of my biggest critics, besides not having Lazy Loading, is, that if we should opt in to Eager Loading using .Include(), then that at least should be implemented consequently meaning, if I don't include a navigation property, then it should never be populated.
As it is right now, my navigation Properties to Parent Objects, at times are loaded even though I did not include anything.
This is confusing and causes nightmares at integration testing as it lets me pass tests, which shouldn't pass, when for example I forgot to include some property. But, as the navigation properties in some strange scenarios are set, those tests pass I only note it, once I user test my application. I'd rather prefer that a navigation property is only set if it was in fact included using include.
These are my 2 cents on EF Core and Lazy Loading. My wish is clearly the possibility to opt in.
@gdoron I don't agree at all when you say "Your entities should not have any methods on them (not even ToString IMHO). When you mix your DAL with your BL like that, as you see, there're troubles."
First of all, POCO classes mean "Plain Old CLR object" classes, that means that your entity class shouldn't be contaminated by any infrastructure classes/namespaces, like you shouldn't derive your entity class from any framework like EF or NHibernate or any ORM. That's it, it has to be pure and clean C#. Period. POCO doesn't mean that you cannot have logic/methods as part of your class. That's not right. On the other hand, if you are following Domain-Driven Design patterns (best approach for large applications with a lot of ever-chaging business rules) in DDD you should have logic as part of the entities, most of all in the aggregate root. This is called "Rich Domain Model" whereas if you just have Data entities, only data, then it is called "Anemic Domain Model" which is not good for very large applications with a lot of business rules. Just search for "Anemic Domain Model" and DDD, Eric Evans, Martin Fowler, etc. and you'll get a lot of info about it. Of course, the logic within the entities should deal ONLY with the data in the entities, it should not access the DB or any service, for sure. That should be part of the application layers, Repositories, etc.
If lazy loading is to be considered a bad idea categorically, then there should be no Lazy
This is an example from the MS Sharepoint List.Items property documentation:
When you call the Items property, it returns an instance of an SPListItemCollection object that does not contain any data, but on first access to an item from the collection, the entire collection object is filled with data. Consequently, to improve performance it is recommended that you assign the items returned by Items to an SPListItemCollection object if you must iterate the entire collection, as seen in the example. It is best practice is to use one of the GetItem* methods of SPList to return a filtered collection of items.
Even though there is a GetItem* method, Microsoft provides a "Items" property.Many Sharepoint developers use this property without having read that documentation. The ones who have read the docs use it intentionally when they are sure that there are just a bunch of items in a list and it won't create a disaster.
We use lazy loading everywhere, even when we are not aware that there is lazy loading.
So, EF is not the first and will not be the last tool which may cause decreased performance or fatal problems when its users don't use it as described in its manual.
I think, Lazy loading is a "must have" rather than a "nice to have" but if it won't be there anyway, the reason of not supporting it must be a pragmatical one rather than a philosophical one.
@ckalan IMHO "lazy loading" was a poorly chosen name for what was happening, therefor the comparison with the Lazy class is wrong. A better name for this technique would be "automatic loading" or maybe even "magic loading". Not being aware of additional round trips to the database is from another magnitude than creating an instance when first accessed in memory.
@riezebosch No, lazy loading is exactly what this should be named. I mean we are not talking here about something magical and new. This is a technology used in a lot of frameworks and its quite common.
@riezebosch I see what you mean but actually most API's use Lazy class in the background and usually the client is unaware that there is a Lazy implementation behind. "In principle" lazy operations are all same. It's just some are "relatively" slow and some are "relatively" fast. Lazy operations behind public API's is a common programming concept and even .NET helps us do that with such helper classes. Therefore a philosophical discussion on whether lazy loading is good or bad is just a dead end in which we have just been involved :) So I suggest we change the question from "Should we support lazy loading?" to "With what kind of a API should we support it to minimize the problems?"
Enabling granular configuration in addition to a global one with a simple attribute on a property or class like [LazyLoadingEnabled(true|false)]
may be an option. This way the developer can decide which classes or properties can be lazily loaded and takes the responsibility by explicitly making a choice. We were able to do this in Java Hibernate and I guess it was also possible with NHibernate.
Note:
With regard to feedback, I think it is worth reiterating some comments made a few months ago. We read and consider all feedback (and try to make the right choices based on it) no matter how it is delivered; that is, whether it is provided politely and with respect or not. Yelling at us or others with different opinions at best makes you feel good and us/them feel bad. It doesn’t achieve anything concrete. We are doing our best to prioritize features and implement them in the best way we can. We really appreciate your feedback and it makes a big difference on shaping the product. We personally appreciate it more when feedback is delivered in a respectful way, but please don’t stop providing constructive feedback.
Original issue:
Hi,
i was wondering if I am the only one that thinks EF Core is useless without Lazy Loading? Or do I do something wrong? Lets just consider a simple scenario when a course provider cancels a course on a course booking plattform written in MVC.
The user calls courses/cancel/1. The action would get the course and call cancel method like here
The cancel method then needs to cancel each booking on the course so it would do something like this
The booking in turn would refund the transaction
So for this to work, lazy loading is needed?! (The solution would be to eager load all the data on the controllers action method. But I do not know what will be needed from the controller?)
I'd appreciate any information on that.