Events and interception (aka lifecycle hooks)

[divega]

Done in 2.1

• [x] Add events for entity state changes #10895

Done in 3.1

• [x] Add IDbCommandInterceptor or similar to allow injecting custom query execution service #15066
• [x] IDbConnectionInterceptor and IDbTransactionInterceptor

Done in 5.0

• [x] Implement events for before and after SaveChanges #15910

Done in 6.0

• [x] Interception: raw query vs. LINQ #22899

Done in 7.0

• [x] Implement ObjectMaterialized event #15911
• [x] ChangeTracker: New event for "entity about to get tracked" #27093
• [x] Don't check for a connection string until after ConnectionOpening has been called #23085
• [x] Add interception event to be fired when EF has finished consuming a result set #23535
• [x] Utility of DataReaderDisposing logging/interception #24295
• [x] Please consider adding two methods to the DbConnectionInterceptor Creating/Created #23087
• [x] Consider delaying DbCommandInterceptor.CommandCreated until DbCommand properties are intialized #17261
• [x] Interception for DbUpdateConcurrencyException #28315
• [x] Interception to modify the LINQ expression tree #28505
• [x] Provide event that can be used to get notifications when all property changes have been detected #26506

Backlog

• [ ] Add interception for transaction enlistment #16261
• [ ] Add interception for execution strategies #16257
• [ ] Add interception for batches #16260
• [ ] Implement interceptors for Cosmos #26478
• [ ] Add an interceptor to an existing DbContext instance #20273
• [ ] Ability to register IInterceptor without an IDbContextOptionsExtension #21578
• [ ] Interceptors for SQL generation #19748

• [ ] Interception for model building #31206

Note: below is a copy of a very old EF specification and reflects thinking from several years ago. A lot of things aren't valid anymore.

We define EF Core lifecycle hooks as the general feature that enables an application or library to sign up to be invoked or notified whenever certain interesting conditions or actions occur as part of the lifecycle of entities, properties, associations, queries, context instances, and other elements in the Entity Framework stack.

For example:

1. An application can provide a method that will be invoked automatically whenever an object is about to be saved, or it can subscribe to an event that fires when an object is created and its properties initialized, etc.
2. A framework extension can register an interceptor that gives it an opportunity to rewrite query expression trees before they get translated by EF. This could be used to validate whether a user has access to specific information or to filter query results based on per DbContext filter (see #6440).
3. Execute SQL after a DbConnection is opened (to use features such as SQL Server App Role)

The need for lifecycle hooks

We want to enable customers to write business logic that triggers in the different stages of the lifecycle of these objects, following well factored coding patterns. We also want framework writers to be able to use these hooks to extend EF Core in useful ways.

In previous versions of Entity Framework we already exposed a few lifecycle hooks. For instance, we had the AssociationChanged and ObjectStateManagerChanged events since the first version, and the ObjectMaterialized event was added in EF4. Up until EF6.x many of the existing hooks are not exposed in the DbContext API. In EF6 we also added several low level extensibility points in Interception that can be used too as lifecycle hooks.

There is a continuum of capabilities related and overlapping with lifecycle hooks, e.g.:

• We want to improve everyone’s ability to diagnose functional and performance issues with their code using Entity Framework by recording information about interesting events and conditions in the EF stack, but that is essentially logging (tracked in #218 and other work items). By comparison, lifecycle hooks allows to modify the outcome of these events.
• Also the DI-based architecture of EF Core allows for a very granular capability to wrap and replace individual implementation services, which can potentially be used to extend EF and to execute business logic. By comparison, lifecycle hooks is about adding simple and first class hooks that can easily be used to react to interesting conditions without having to re-implement the complete interface of a service.

  Target customers

  1. Application developers that need to implement business logic that is triggered at certain points in the lifecycle of an application, for instance, before an added or modified object is saved to the database, just before loading the contents of a navigation property, etc. Someone could add an OnValidate method to an entity that gets executed right before the entity is stored.
  2. Framework/tool writers that need to extend the behavior of Entity Framework to support new application scenarios or to integrate it with other products or frameworks can write their own frameworks on top of EF that customize the behavior, collect data about the execution of specific actions, etc. Someone can write a full-fledged profiler taking advantage of the hooks.

     Goals & Principles

  3. Should work well and be consistent with DbContext design principles
  4. Should have a consistent story on when and how it is allowed to re-enter: With a very small set of rules users should be able to predict what is allowed and what is not. The stack should be resilient to reentrancy in some cases and throw good exceptions if reentrancy occurs in other cases. We should prevent doing the same work more than once.
  5. Should have minimal performance impact, especially when hooks are not being used the impact should be insignificant.

     A brief survey of hooking mechanisms

     There are not only different interesting conditions or actions an application may need to listen to, but also different kinds of mechanisms to implement hooks that present distinctive characteristics along the following dimensions:

  6. Compile-time vs. run-time binding
  7. Performance
  8. Cancellability
  9. Override/customization of default behavior
  10. Single vs. multi-cast
  11. Simplicity
  12. Discoverability
  13. Familiarity Note: We are looking for criteria that will help us choose the best type of hook for each extensibility point, as well as for the possibility of defining unified hook mechanisms that we can leverage to support different patterns with the same framework code.

      .NET Events

      Events are the most common hook pattern that almost every API in .NET uses. Events are messages sent by sender object to one or more receiver objects through a multicast delegate that acts as a dispatcher. Among the characteristics of events, they support runtime subscribe/unsubscribe, multiple listeners, and are relatively easy to use. Events can be slower than other hook mechanisms, but they have the advantage of being very discoverable (they are usually public members on the sender object) and familiar to customers. Events also provide a standard way to model cancellable actions with CancelEventArgs.

      Virtual methods

      Virtual methods require the application code to declare a derived type and override the method. Virtual methods provide better performance than events but are slower than regular method invocation. Virtual methods are easy to discover and provide a very nice model for overriding/customizing default behavior and chaining with subsequent derivate types. Visual Studio provides a nice Intellisense experience for virtual methods: when you write the overrides keyword in C#, Intellisense provides the list of all the virtual methods available.

      Delegates

      A more efficient alternative to events, regular (non-multicast) delegates can also be used as a hook. Users can normally provide some implementation of a predefined delegate signature (usually a Func<T…> or Action<T…> that can be implemented as a regular method, and anonymous method or a lambda expression) as a parameter to a framework method, as the return type from a method or as a property. Then the delegate is invoked by the framework at appropriate times. While delegates are often compared to strongly typed function pointers, they in fact are very flexible with regards to the signature (they support variance).

      Partial methods

      Partial methods were introduced in .NET 3.5 as a means to extend generated code in separate partial classes. Partial methods are void methods that are both defined and invoked in the right places in generated code. Users can choose to provide the implementation of partial methods in a separate partial class, and the compiler will resolve the partial method to the implementation provide by the user. When the implementation of a partial method is not provided, all calls to the method and its definition are removed by the compiler. Since partial methods are either turned into regular methods or removed, the mechanism is extremely efficient. Partial methods are discoverable because Visual Studio provides a nice Intellisense experience for them. Similar to virtual methods, once you write the keyword partial inside the partial class, Visual Studio editor will list all the partial method definitions that haven’t been implemented.

      Magic methods

      The concept of magic methods is that the user can write a method that follows a particular naming convention and signature, and a framework component will make sure the method will get invoked automatically at runtime. Usually an expression is compiled at runtime to produce a delegate that be used to invoke the method multiple times very efficiently. There is some runtime overhead in compiling the expression, but this is paid only once. Magic methods can be instance or static method. A common practice for magic methods, when code generation is involved, is to provide the declaration of the magic method as a partial method. Although no actual calls are generated, the partial method that gets an implementation will be compiled into the assembly so that it can be invoked at runtime. The only reason for this is the Intellisense experience you get. Magic methods can be used to override default behaviors but it is necessary to expose a public method that implements the default behavior so that the user has the option to invoke this method from within the magic method if he doesn’t want to completely override it. LINQ to SQL uses this mechanism pervasively.

      Attributed methods

      Similar to magic methods, an instance or static method with the right signature can decorated with a special attribute that specifies a runtime role for this method. The runtime examines types for the presence of these attributes and registers the method to be executed on the occurrence of certain conditions. WCF Data Services uses this pattern pervasively.

      Listener interfaces

      This approach consists on defining a class that implements a custom interface provided by the framework and at a later point register an instance of this class as a listener for particular events. Usually, the interface defines one or more methods with a very specific purpose, but the listener class can be a composite of multiple interfaces. Discoverability can be improved in this programing model with a base listener interfaces and by placing all the related interfaces under the same namespace.

      IObservable<T>

      In many situations a mechanism is required to handle a stream of asynchronous events coming from the same source. IObservable<T> is an analog to IEnumerabe<T> that can be used to represent this kind of source.

      Context hooks vs. entity and property hooks

      DbContext provide a very good place for us to focus when defining extensibility hooks for anything that has to do with the functions that they encompass:

  14. Store Connection
  15. Launchpad for queries
  16. Unit of work
  17. Update adapter But for hooks that are specific to the lifecycle of entity or complex types, properties, etc., other options exist:
  18. Factor business logic into the entity type itself: The typical example of this is the OnValidate method that LINQ to SQL supports: a user can define the method on a particular entity type, and have the framework will automatically invoke the method at appropriate times.
  19. Extend the Code-First API to support hooks: The Code-First API provides a nice central configuration point for entity types, complex types and properties. We could extend the API with methods to register listeners, e.g.:

      mb.Entity<Product>().Notify(new EntityLoadedListener());
      mb.Entity<Customer>().Property(c => c.Orders).Notify(new PropertyLoadedListener());

      The one potential issue with this approach is that the hook configuration would become part of the model and therefore it would not be possible to change it once the context object has been instantiated. This might be an acceptable limitation however, since any necessary changes in behavior can be coded into the listener class itself. Making the configuration of hooks immutable also has the advantage of allowing for compiling the invocations to listeners into efficient delegates.

      EF requirements for a hooking mechanims

      We are trying to find a design that has the following characteristics:

  20. Works well for both app developers and framework developers.
  21. Makes hooks very discoverable.
  22. Invocation is efficient.
  23. Requires only very simple coding patterns to use.
  24. Should allow get the hooks at runtime and from a separate assembly.
  25. Hook handlers can be implemented outside the entity types.
  26. Hook handlers can be implemented as part of the entity types.
  27. Ideally, the way you implement a hook handler in an entity shouldn’t break POCO.
  28. The complexity of supporting multiple hook mechanism should be completely hidden from the sender. This requires further analysis, but it seems that it would be reasonable to support a mix of hook mechanisms (but not too many) to optimize for different scenarios. The current thinking is that we would use a single generic class to represent a hook. The sender should just need to call a method or instantiate a class and call a method, nothing much more complex than the typical “On[EventName]” pattern, and the hook class would make sure all the hook handlers are invoked. We are also considering building a convention system for wiring up hook handlers in the entity types. When designing this we should take advantage of any opportunity to reuse some hook mechanism as building blocks for others. For instance:
  29. If we decide that the basic mechanism is listener interfaces, then we can have context objects implement those interfaces and re-cast the hooks as events.
  30. We can have a single convention system that understands about naming patterns and attributes (similar to how the Code-First convention system understands naming patterns and data annotations as different vocabularies to express the same kind of concepts).  

      Lifecycle hooks list

      The following is an incomplete list that presents various hooks we could consider adding. Intentionally the list does not try to be specific on each hook about certain details:

  31. Hooking mechanisms
  32. Method / argument signatures
  33. All names of new hooks are up for discussion

Name	Pri	Location	Cancel or override	Description & sample scenario
QueryExecuting	0	DbContext	Yes	Query interception, custom query caching.
QueryExecuted	3	DbContext	No	When Execute happened, before the reader is read. Tracing?
QueryCompleted	3	DbContext	No	After DbDataReader is closed. Tracing?
EntityStateChanged	0	DbContext	No	Signals all state changes
EntityStateChanging	3	DbContext	?	Undo changes or change proposed values before they are set?
ConnectionProvisioning	2	DbContext	Yes	Execute additional code to make sure the connection is alive, or do logging
ConnectionReleasing	2	DbContext	Yes	Cleanup something done during Ensure / StartUsingConnection
ConnectionOpened	2		No	More likely for tracing. Since SqlClient has fixed invalid connection pools, then this is lower priority
ConnectionOpening	1	DbContext	Yes	Slightly simpler to use than Ensure/Start, would not require user to check current state. Could also be used for tracing.
ConnectionClosed	1		No
ConnectionClosing	1	DbContext	Yes	Slightly simpler to use than Release/Stop, would not require user to check the initial state. Could also be used for tracing.
OnModelCreating	0	DbContext	Yes	Tweak model before it is cached.
OnModelCreated	1	DbContext	Yes	Signal that the model is done and execute some custom code, possibly related to caching logic. . Issue: do we need this for ObjectContext? Issue: if the user is going to implement his own caching, we should have an abstract class or interface for that.
ModelCacheLookup	2	DbContext	Yes	Implement your own caching logic. Tracing?
ModelCacheHit	2	DbContext	Yes	Execute additional code when the model is found in the cache. Tracing?
EntityLoading	1	DbContext, DbEntityEntry	No	After object instance is created but before its properties are initialized. Can be used to reset a flag that will be set in newly created instances but shouldn’t be set during initialization, i.e. for validation.
EntityLoaded	0	DbContext, DbEntityEntry	No	Can be used to setup anything after an object has been materialized, i.e. event handlers, flags, etc.
CollectionLoading	1	DbContext, DbEntityEntry DbCollectiohnEntry	No	Can be used to setup anything on a collection after it is created but before it is populated. Issue: Could be used to provide your own collection?
CollectionLoading	1	Context, Entity or Collection	No	Can be used to setup anything on a collection after it has been created and populated, i.e. listeners for its changed event.
ObjectTypeResolving	1	Context	Yes	Could be used to specify a different type than the original one, i.e. to implement your own proxy mechanism. It should be per type but could return a Func<T> that returns a new instance and the result could be compiled into materialization delegates.
CollectionTypeResolving	1	Context	Yes	Something similar to ObjectTypeResolving but for collections. Could be used to replace the default collection type with a custom proxy collection with additional functionality (i.e. paging, fine grained lazy load).
Virtual OnSavingChanges	Medium	DbContext	No	Can be used to re-implement SaveChanges but still invoke the existing SavingChanges event
SavedChanges	Low	Context	No	Could be used to execute cleanup code after SaveChanges. For instance, to call AcceptChanges on each STE change tracker. It is lower priority because virtual SaveChanges covers most scenarios.
EntityStateChanging	Low	Context, Entity	Yes	For an entity instance or type in particular we could avoid putting in the modified state. So even if the properties are read-write, the context ignores changes to this entity. Could be also used to suspend fixup on an entity that is being detached.
EntityStateChanged	High	Context, Entity	No	Executes logic after an entity has been put in a certain state. Can be used to setup property values, restore state after the changing event.
PropertyChanging	Low	Context, Entity	Yes	Any time a property is about to be changed by the framework or any party, if notification or interception is enabled by the entity type. Should make original and new value available. Should also work for navigation, scalar and complex types properties. Tracing?
PropertyChanged	High	Context, Entity	No	Any time a change in a property value change is detected.
PropertyLoading	High	Context, Entity, Collection	Yes	Intercepts, overrides de loading of a property. Could be used to support loading of properties using stored procedures.
PropertyLoaded	Medium	Context, Entity, Collection	No	Tracing?
Writetable IsLoaded	High	Context, Entity	Yes	Allows cancelling the loading of a property.
CollectionChanging	Medium	Context	Yes	Any time a collection is about to be changed by the framework or any party, if interception is enabled
CollectionChanged	Medium	Context, Entity	No	Any time a change to a collection has been detected.
AssociationChanging	Low	Context, RelatedEnd	Yes	Can be used to prevent an association from being changed, or to execute business logic when the association is about to change.
AssociationChanged	Medium	Context, RelatedEnd	No	Can be used to execute additional logic after an association is changed, i.e. user can explicitly cascade relationships removals into dependent removals, workaround current databinding shortcomings.
RowValidate , RowValidateAdded, RowValidateModified, RowValidateDeleted	Medium	Context	Yes	Storage level version of ObjectValidate. Tracing?
SavingChanges event	0	DbContext	?	Currently only available on ObjectContext. Should make trigger OnSavingChanges method protected.

Existing hooks

Name	Description & sample scenario
virtual Dispose	This can be used to do additional cleanup, i.e. on entity instances.
virtual SaveChanges	Can be used to execute additional logic before, after or instead of saving changes.

Some open issues:

1. Need to prototype some coding patterns and try them.
2. Is logging and tracing part of this API? It seems that ideally we should have the same level of flexibility for hooking mechanisms with logging and tracing as we end up having with this API.
3. Second level cache should probably expose its hooks through the same mechanisms.
4. Should we provide low level query interception points with the same mechanisms, i.e. as command tress and store commands? Should we do the same for CUD store commands? Would need to make sure those work well with caching.
5. Can we get some level of support for async execution of queries with this hook?
6. Should we provide enough lifecycle hooks to implement custom fixup logic?
7. Areas of overlap with other extensibilities: read and write properties in object mapping, proxy type creation (can be imperative vs. event driven), equality and snapshot comparisons for change tracking extensibility.
8. What about customizing identity resolution?
9. Is Logging and Tracing part of the lifecycle hooks
10. Is Query interception part of the lifecycle hooks
11. Even without query interception we should expose when we are about to execute (imagine a profiling tool that measures how much query compilation costs).
12. Is ContinueOnConflict part of the lifecycle hooks
13. How do we improve diagnostics? Can we have OnError
14. Need to do prioritization, costing and scoping
15. Should we have a fine grained version of CollectionAdding / CollectionRemoving with support for magic methods on the entities to enable collection patterns? We would need a pattern for Contains checks also.
16. There is a conversation about splitting AssociationChanged this event into properties and collection changes. However, there should be a way to tell the difference between a scalar property change and a nav prop. Should we make AssociationChanged more accessible and add AssociationChanging? This would provide a way to intercept changes in associations independently of cardinality, constraints, etc.
17. AssociationChanging would need the entity and collection types to collaborate to avoid changes from being made to the graph.

[SidShetye]

@divega and @ajcvickers : it's been several months since an update so wanted to ask if you have clarity on ObjectMaterialized and SavingChanges hook release dates?

[ajcvickers]

@SidShetye Unfortunately, the schedule for 3.0 is pretty full, and this is not planned for that release--as indicated by being in the "Backlog" milestone. That being said, it's possible something small will get slipped in depending on how things go.

[optiks]

For those that are blocked by A second operation started on this context before a previous operation completed. Any instance members are not guaranteed to be thread safe, we worked around this by delaying our custom ObjectMaterialized-like event in a few different places.

High-level steps are: 1) For queries that can return multiple results (e.g. Where, Select, ...):

• Implement IEnumerable<T> with a custom Enumerator
  • Raise the custom event during MoveNext recursively

2) For queries that can only return one results (e.g. FirstOrDefault, Single, ...):

• Implement IQueryable<T>
• Raise the custom event during Execute recursively

3) For lazy loading:

• Call ILazyLoader.Load(...)
• Raise the custom event recursively

NOTE: You also need to keep track of whether the entity has already had the event raised somewhere, otherwise you could end up raising the event multiple times.

Happy to share an implementation somewhere if people are interested.

[WillSullivan]

@ajcvickers So, let's say someone wanted to rewrite queries (to implement complex filtering not supported by the built in filters), where would you suggest they insert themselves into the process?

There seems to be ample opportunity to do this (for example, the Expression passed to the QueryCompiler looks so tempting, and I'd like to beat the query cache with a baseball bat), but lots of warnings not to, and a corresponding lack of documentation on the internals.

A little guidance would be appreciated. I've been investigating and prototyping rewriting queries for the past month and have come up agonizingly close to a practical solution.

[ajcvickers]

@WillSullivan We don't have any great general documentation here. It's something I would like to improve, but unfortunately resources are limited, especially considering we only have engineering resources to write docs. If you have specific questions, then please file a new issue with those questions and we'll try to answer.

[WillSullivan]

@ajcvickers uh, "So, let's say someone wanted to rewrite queries (to implement complex filtering not supported by the built in filters), where would you suggest they insert themselves into the process?"

[smitpatel]

@WillSullivan - If you want to do before any kind of query execution starts, then override IQueryCompiler. If you want to change only when query is being compiled then override IDatabase.CompileQuery* methods.

[weitzhandler]

I need to run certain operations on the database when it got created. Is there a way to get a notification when it's creation completed (i.e. after calling EnsureCreated created the DB)?

[ErikEJ]

Is this for testing? EnsureCreated should not be used in production.

[weitzhandler]

Thanks for your notice @ErikEJ, my question is also about context.Database.Migrate(). Is there a way to intercept those calls and perform stuff afterwards?

[smitpatel]

@weitzhandler - How about writing custom SQL in migration files? After database is created, the first line executed on server would be first line in up method of very first migration file

[weitzhandler]

@smitpatel @ErikEJ I've opened a new issue.

[SidShetye]

Hi folks - any news on when this will make the release? Been requested since 2014, now 2019. 2019 - 2014 = 5 :(

[Igneous01]

It's sad that EF still doesn't support this when Hibernate has had this capability for many years now.

If I want to add basic auditing and Guid generation I have to hand roll it myself each time.

This should be something that's possible within EF.

` public class Audit {

@Column(name = "created_on")
private LocalDateTime createdOn;

@Column(name = "created_by")
private String createdBy;

@Column(name = "updated_on")
private LocalDateTime updatedOn;

@Column(name = "updated_by")
private String updatedBy;

@PrePersist
public void prePersist() {
    createdOn = LocalDateTime.now();
    createdBy = LoggedUser.get();
}

@PreUpdate
public void preUpdate() {
    updatedOn = LocalDateTime.now();
    updatedBy = LoggedUser.get();
}

//Getters and setters omitted for brevity

} `

[mguinness]

@Igneous01 You can already achieve this in EF Core - see Generate tracking columns for more details.

[jakenuts]

SaveChanges seems like a great place to be able to intercept/extend a context (auditing, caching events) and now with DbContextPooling overriding those methods to call services is less manageable as they can't be passed in to the constructor. A BeforeSaveChanges(Async), AfterSaveChanges(Async) and ErrorSavingChanges(Async) event would be awesome.

[ajcvickers]

@jakenuts A couple of points:

• If pooling is limiting what you need to do, try switching pooling off. In most scenarios the performance gain is negligible.
• I'm not sure these events would help that much over SaveChanges in this respect, since you still need to register the events, and if there is a convenient place to do that, then why not just set context state there instead.

[sjb-sjb]

I would support the events mentioned by @jakenuts. I implemented very similar events in a wrapper class as a way to update entity view models.

A very handy function to complement SaveChanges would be RevertChanges, with somewhat similar events associated with it. One typically wants to revert changes when a user cancellation occurs.

sjb

[ajcvickers]

@sjb-sjb Not saying those aren't useful events. They are still on the table.

[SidShetye]

Hi Arthur and Diego - can we raise the priority of the lifecycle hooks? The community demand has been very strong (30+ participants, 50+ thumbs, over 5+ years) but this item has been waiting (waiting, waiting ...) in the queue to be punted across every release. I understand the team has a challenging schedule but perhaps a swap/re-prioritization against another issue/feature would help get this long-waiting item out of the queue? Thanks

CC: @ajcvickers @divega

EDIT: More precisely, the worry is this isn't on the 3.0.0 milestone or on any plan in general.

[ajcvickers]

@SidShetye Unfortunately, at this point there isn't anything that can be bumped to give this a higher priority.

[douglasg14b]

Hell, lifecycle hooks have been a community want since EF v4 in 2010.

[SidShetye]

@ajcvickers That is disappointing - and hard to believe. Why? Because looking at the 474 issues open in 3.0.0 milestone, there is only ONE. solitary. single. issue with more customer demand than this issue. Every one of the remaining 473 issues has less customer expressed demand or importance (!).

I mean your customers are literally telling you "this issue is more desired than the other 99.78% of the issues". And yet the product road-map simply tosses this aside? Even with noisy measurements, the balance is overwhelmingly off.

I believe product development should be rationally driven by the customer and markets. But this issue gives the impression that it's highly whimsical. Like: "Yeah, you customers can go take a hike, we're going to build whatever we want." On top of it, this is also nothing new (since you said "at this point") - we've been at it since 2014!

We strive to work closely but this is a disappointing project dynamic.

CC: @divega

[divega]

@SidShetye Customer feedback is very important to us. But there are several other sources of information we use to decide what goes into each release. By the way, we made an attempt to describe the process at https://docs.microsoft.com/ef/core/what-is-new/roadmap#release-planning-process.

Besides that, this isn't really a regular issue. It is more of an "umbrella" issue that represents several individual features. Each lifecycle hook clearly should have its own priority, but because we haven't broken it down into individual issues (something I would like to do, but I haven't found the time for), we get lots of votes here.

We actually implemented some of the features that used to be tracked here (state change events) in EF Core 2.1.

You are welcome to comment here about what specific lifecycle hooks would be most valuable for you and why.

You can offer to help by contributing some of them.

You are also welcome to complain about the fact that your favorite features aren't implemented and all the customers that are tracking this issue will see it. But let me set the expectation that it is very late in the 3.0 release, and given the number of work items we have in progress and the resources we have available, it is very unlikely that we will be adding any more enhancements to the list of this release.

[SidShetye]

@divega @ajcvickers - believe it or not, we're huge fans of the EF team and respect the work coming out right from EFs early days. So much so, that at the unique intersection of ORMs and Data Security where our Crypteron products, services and IP reside, we've heavily invested and have always treated EF (till EF 6.x) as our 'golden target' that drive the designs for other ORMs we support like Java's JPA/Hibernate. We're also promoting enterprise migration to the cloud/Azure by reducing data security concerns on those workloads - these are common customers between your team and us. In fact we're an Azure Security Partner working on an even closer commercial integration. The synergies are great and we're EF proponents in general.

Mapping to some of the guiding questions on your roadmap planning process:

• Guide 1: How many devs? => The votes here are a near landslide
• Guide 2: Workarounds? => Our service has users across languages and libraries but without the extensibility requested here, it's hard to support EF Core users as smoothly as we would really want. Short of forking EF, there isn't a workaround.
• Guide 4: Extensibility point? => This is one of the strongest extensibility points I can think of
• Guide 5: Synergies? => This builds synergies with products of one of your partners (us). We in turn tie in with Azure, Key Vault, VS/Nuget and several other Microsoft technologies.

In terms of what lifecycle hooks are desired, why etc., we discussed that here last year. I hear you about challenges with time on 3.0.0. You guys know best about internal resource allocation, but perhaps 3.0.1 :) ?

Thanks sid(at)crypteron.com

[AFDevMike]

I very much appreciate the EF teams work and am hesitant to step into what is best described as a passionate back and forth. I have one request however.

• This 'issue' is acknowledged as having the 2nd most customer demand.
• A counter point is, this issue is an umbrella issue whereby "Each lifecycle hook clearly should have its own priority"

Objectively, I would assert if this umbrella issue has the 2nd most customer demand, but can't be treated as a single feature request. Its worthy of the time to break into individual issues to track demand more accurately.

[divega]

@SidShetye thanks for the pointer to https://github.com/aspnet/EntityFrameworkCore/issues/626#issuecomment-359082350. To give you some perspective, there were ~90 comments before that, and ~70 comments after that in this issue. That is one of the reasons it has become extremely hard to distill the comments in this issue into actionable work items that we can prioritize. I would be very grateful if you go ahead and create a new issue for the specific hooks you need. Please, try to be very specific about what you need (unfortunately, after re-reading that comment a couple of times, I am still not sure), and feel free to link to it from here.

@AFDevMike

I would assert if this umbrella issue has the 2nd most customer demand

I wonder how you are counting. In https://github.com/aspnet/EntityFrameworkCore/issues?q=is%3Aissue+is%3Aopen+sort%3Areactions-desc this appears to be in the 14th position. But as I said, the main problem is that it is impossible hard to distill what it really means in terms of actionable work items for the individual lifecycle hooks and their relative priorities.

[AFDevMike]

@divega I was counting based upon @SidShetye's assertion here: https://github.com/aspnet/EntityFrameworkCore/issues/626#issuecomment-496730616

Because looking at the 474 issues open in 3.0.0 milestone, there is only ONE. solitary. single. issue with more customer demand than this issue. Every one of the remaining 473 issues has less customer expressed demand or importance (!).

And since it wasn't refuted in your direct response, I assumed it to approximate the reality. My error, apologies for noise.

[weitzhandler]

@divega

I opened a new issue that is related to life-cycle hooks. In my scenario, I need to have a post-OnConfiguration hook, so I can call Database.Migrate. See #15846.

[ajcvickers]

See #15066 for IDbCommandInterceptor (for command interception), for which there is now a PR: #16113

[SidShetye]

For folks tracking this directly or coming from search (SEO!) - the actual work items are in #15910 and #15911. Hoping to see this released in EF Core 3.1 🤞 !

[RobK410]

We need interception to rewrite the SQL before execution because there is no support in EF for FULLTEXT functions nor XML XPath query syntax. We dynamically build an IQueryable based on an ad hoc query builder interface in our application, and that turns our XML data typed model properties into column LIKE '%value%' statements in the SQL.

Interceptors were the ONLY work around we could come up with to parse the SQL, replacing the LIKE '%value%' with CONTAINS(fulltextcolumn, 'value') and xmlcolumn.query('xpath value') = 1

Where's the common work around described in this opened issue for manipulating the SQL before execution?

I'd personally like to see full XML and FULLTEXT syntax support. Is there a issue tracked for those? Can the community code and submit these changes?

[ajcvickers]

@vasont For full-text search see #10488, #11481, #11486. For the low-level hook to modify SQL, database interceptors is the way to do this.

[RobK410]

@vasont For full-text search see #10488, #11481, #11486. For the low-level hook to modify SQL, database interceptors is the way to do this.

@ajcvickers Thanks so much for the references. A project this big can get unruly trying to find the right info with different search terms.

Awesome news on the FreeText search functions. It would be more awesome to see similar for the XML query functions. :)

[douglasg14b]

@vasont I'm happy I'm not alone in finding information discovery challenging with large projects like this!

@ajcvickers Which is why I (and I'm sure many others) greatly appreciate this repos openness to question issues. As opposed to offloading those to a gitter or chat where the information is rapidly lost to time. It makes it possible to find conversations and solutions to niche issues that would otherwise be neigh-impossible to locate or discover, and gives way for some niche discussions to turn into features.

[Nefcanto]

Is this still not created in EF Core?

[mguinness]

@nefcanto Take a look at interception of database operations in New features in Entity Framework Core 3.0, but lifecycle hooks didn't make it into 3.0 release.

[JValck]

For those who came here via SEO: because I needed this functionality myself, I created my own package with lifecycle hooks.

You can download it via NuGet. Full docs are available on the Github page of the project.

[jakenuts]

I was just wondering how I might detect changes to a relationship collection to update the owner and it seems like this is the same issue. You've built a great system for loading and persisting data but between those two actions there is "change" and it is only addressed in terms of the first two. When I modify a property, add an item to a collection I want an easy method for hooking into that process and reacting. We've probably all written some complicated method of using change tracking to do tiny little tasks like setting an UpdatedDate and that just promotes brittle redundant code across all of our projects. Prioritize change hooks, please.

[sjb-sjb]

@jakenuts you can use notification tracking entities with a collection that implements INotifyPropertyChanged/Changing, see https://blog.oneunicorn.com/2016/11/16/notification-entities-in-ef-core-1-1 You can also use a collection type that implements INotifyCollectionChanged.

In this approach, when you assign to a principal entity reference and both entities are attached to the same context, EF will immediately add the dependent entity to the parent collection resulting in a CollectionChanged from the collection.

Watch out for the fact that this only happens when both entities are attached to the same context. In my work I attach the entities if they are not already attached.

You can also expect to get PropertyChanged notifications for database-assigned primary keys when they are set during a SaveChanges.

[chester89]

We also need to propagate changes to several systems. I looked at @JValck solution, found it too fine-grained for our purposes. At the end I implemented simple publish/subscribe by overriding DbContext.SaveChangesAsync method. It would be great to have such mechanism in the library itself, because IDbCommandInterceptor API is too low-level

[markusschaber]

Propagating changes in a transaction safe way is not easily possible - the service might crash in the wrong moment, leaving an inconsistency between the propagated changes and the real database contents.

We're thinking about using PostgreSQL LISTEN and NOTIFY mechanism for this problem.

[optiks]

Propagating changes in a transaction safe way is not easily possible - the service might crash in the wrong moment, leaving an inconsistency between the propagated changes and the real database contents.

We're thinking about using PostgreSQL LISTEN and NOTIFY mechanism for this problem.

I’d suggest you’d be better going with with one:

• publish with later polling, or
• DTC, or
• a mechanism that can mimic DTC like the NServiceBus Outbox (https://docs.particular.net/nservicebus/outbox/)

[chester89]

Propagating changes in a transaction safe way is not easily possible - the service might crash in the wrong moment, leaving an inconsistency between the propagated changes and the real database contents. We're thinking about using PostgreSQL LISTEN and NOTIFY mechanism for this problem.

I’d suggest you’d be better going with with one:

• publish with later polling, or
• DTC, or
• a mechanism that can mimic DTC like the NServiceBus Outbox (https://docs.particular.net/nservicebus/outbox/)

DTC is not supported in .NET Core https://github.com/dotnet/runtime/issues/715

[markusschaber]

@optiks LISTEN and NOTIFY in PostgreSQL are integrated with the database transactions, so what's the reason you advise against them?

Publish with later polling puts a high load if you have lots of data, and only small amounts change. Also, you increase latency by the polling interval.

We'll have a closer look at NServiceBus Outbox. But at first glance, it just looks like reinventing a transactional messaging system in the database by storing message in tables, so why not just use the one provided by the database?

[ajcvickers]

See https://github.com/dotnet/efcore/issues/27093

[Evengard]

Is there a OnModelCreating override implemented, or smth along this issue: https://github.com/dotnet/efcore/issues/9330 ?

[ajcvickers]

@Evengard I opened #31206.

[xaviergxf]

is there any interceptor available that allow audit feature when using ExecuteUpdate, ExecuteDelete?

[roji]

@xaviergxf command interceptors should work for ExecuteUpdate/Delete just like for any other EF database operation.