Events and interception (aka lifecycle hooks)

[divega]

Done in 2.1

• [x] Add events for entity state changes #10895

Done in 3.1

• [x] Add IDbCommandInterceptor or similar to allow injecting custom query execution service #15066
• [x] IDbConnectionInterceptor and IDbTransactionInterceptor

Done in 5.0

• [x] Implement events for before and after SaveChanges #15910

Done in 6.0

• [x] Interception: raw query vs. LINQ #22899

Done in 7.0

• [x] Implement ObjectMaterialized event #15911
• [x] ChangeTracker: New event for "entity about to get tracked" #27093
• [x] Don't check for a connection string until after ConnectionOpening has been called #23085
• [x] Add interception event to be fired when EF has finished consuming a result set #23535
• [x] Utility of DataReaderDisposing logging/interception #24295
• [x] Please consider adding two methods to the DbConnectionInterceptor Creating/Created #23087
• [x] Consider delaying DbCommandInterceptor.CommandCreated until DbCommand properties are intialized #17261
• [x] Interception for DbUpdateConcurrencyException #28315
• [x] Interception to modify the LINQ expression tree #28505
• [x] Provide event that can be used to get notifications when all property changes have been detected #26506

Backlog

• [ ] Add interception for transaction enlistment #16261
• [ ] Add interception for execution strategies #16257
• [ ] Add interception for batches #16260
• [ ] Implement interceptors for Cosmos #26478
• [ ] Add an interceptor to an existing DbContext instance #20273
• [ ] Ability to register IInterceptor without an IDbContextOptionsExtension #21578
• [ ] Interceptors for SQL generation #19748

• [ ] Interception for model building #31206

Note: below is a copy of a very old EF specification and reflects thinking from several years ago. A lot of things aren't valid anymore.

We define EF Core lifecycle hooks as the general feature that enables an application or library to sign up to be invoked or notified whenever certain interesting conditions or actions occur as part of the lifecycle of entities, properties, associations, queries, context instances, and other elements in the Entity Framework stack.

For example:

1. An application can provide a method that will be invoked automatically whenever an object is about to be saved, or it can subscribe to an event that fires when an object is created and its properties initialized, etc.
2. A framework extension can register an interceptor that gives it an opportunity to rewrite query expression trees before they get translated by EF. This could be used to validate whether a user has access to specific information or to filter query results based on per DbContext filter (see #6440).
3. Execute SQL after a DbConnection is opened (to use features such as SQL Server App Role)

The need for lifecycle hooks

We want to enable customers to write business logic that triggers in the different stages of the lifecycle of these objects, following well factored coding patterns. We also want framework writers to be able to use these hooks to extend EF Core in useful ways.

In previous versions of Entity Framework we already exposed a few lifecycle hooks. For instance, we had the AssociationChanged and ObjectStateManagerChanged events since the first version, and the ObjectMaterialized event was added in EF4. Up until EF6.x many of the existing hooks are not exposed in the DbContext API. In EF6 we also added several low level extensibility points in Interception that can be used too as lifecycle hooks.

There is a continuum of capabilities related and overlapping with lifecycle hooks, e.g.:

• We want to improve everyone’s ability to diagnose functional and performance issues with their code using Entity Framework by recording information about interesting events and conditions in the EF stack, but that is essentially logging (tracked in #218 and other work items). By comparison, lifecycle hooks allows to modify the outcome of these events.
• Also the DI-based architecture of EF Core allows for a very granular capability to wrap and replace individual implementation services, which can potentially be used to extend EF and to execute business logic. By comparison, lifecycle hooks is about adding simple and first class hooks that can easily be used to react to interesting conditions without having to re-implement the complete interface of a service.

  Target customers

  1. Application developers that need to implement business logic that is triggered at certain points in the lifecycle of an application, for instance, before an added or modified object is saved to the database, just before loading the contents of a navigation property, etc. Someone could add an OnValidate method to an entity that gets executed right before the entity is stored.
  2. Framework/tool writers that need to extend the behavior of Entity Framework to support new application scenarios or to integrate it with other products or frameworks can write their own frameworks on top of EF that customize the behavior, collect data about the execution of specific actions, etc. Someone can write a full-fledged profiler taking advantage of the hooks.

     Goals & Principles

  3. Should work well and be consistent with DbContext design principles
  4. Should have a consistent story on when and how it is allowed to re-enter: With a very small set of rules users should be able to predict what is allowed and what is not. The stack should be resilient to reentrancy in some cases and throw good exceptions if reentrancy occurs in other cases. We should prevent doing the same work more than once.
  5. Should have minimal performance impact, especially when hooks are not being used the impact should be insignificant.

     A brief survey of hooking mechanisms

     There are not only different interesting conditions or actions an application may need to listen to, but also different kinds of mechanisms to implement hooks that present distinctive characteristics along the following dimensions:

  6. Compile-time vs. run-time binding
  7. Performance
  8. Cancellability
  9. Override/customization of default behavior
  10. Single vs. multi-cast
  11. Simplicity
  12. Discoverability
  13. Familiarity Note: We are looking for criteria that will help us choose the best type of hook for each extensibility point, as well as for the possibility of defining unified hook mechanisms that we can leverage to support different patterns with the same framework code.

      .NET Events

      Events are the most common hook pattern that almost every API in .NET uses. Events are messages sent by sender object to one or more receiver objects through a multicast delegate that acts as a dispatcher. Among the characteristics of events, they support runtime subscribe/unsubscribe, multiple listeners, and are relatively easy to use. Events can be slower than other hook mechanisms, but they have the advantage of being very discoverable (they are usually public members on the sender object) and familiar to customers. Events also provide a standard way to model cancellable actions with CancelEventArgs.

      Virtual methods

      Virtual methods require the application code to declare a derived type and override the method. Virtual methods provide better performance than events but are slower than regular method invocation. Virtual methods are easy to discover and provide a very nice model for overriding/customizing default behavior and chaining with subsequent derivate types. Visual Studio provides a nice Intellisense experience for virtual methods: when you write the overrides keyword in C#, Intellisense provides the list of all the virtual methods available.

      Delegates

      A more efficient alternative to events, regular (non-multicast) delegates can also be used as a hook. Users can normally provide some implementation of a predefined delegate signature (usually a Func<T…> or Action<T…> that can be implemented as a regular method, and anonymous method or a lambda expression) as a parameter to a framework method, as the return type from a method or as a property. Then the delegate is invoked by the framework at appropriate times. While delegates are often compared to strongly typed function pointers, they in fact are very flexible with regards to the signature (they support variance).

      Partial methods

      Partial methods were introduced in .NET 3.5 as a means to extend generated code in separate partial classes. Partial methods are void methods that are both defined and invoked in the right places in generated code. Users can choose to provide the implementation of partial methods in a separate partial class, and the compiler will resolve the partial method to the implementation provide by the user. When the implementation of a partial method is not provided, all calls to the method and its definition are removed by the compiler. Since partial methods are either turned into regular methods or removed, the mechanism is extremely efficient. Partial methods are discoverable because Visual Studio provides a nice Intellisense experience for them. Similar to virtual methods, once you write the keyword partial inside the partial class, Visual Studio editor will list all the partial method definitions that haven’t been implemented.

      Magic methods

      The concept of magic methods is that the user can write a method that follows a particular naming convention and signature, and a framework component will make sure the method will get invoked automatically at runtime. Usually an expression is compiled at runtime to produce a delegate that be used to invoke the method multiple times very efficiently. There is some runtime overhead in compiling the expression, but this is paid only once. Magic methods can be instance or static method. A common practice for magic methods, when code generation is involved, is to provide the declaration of the magic method as a partial method. Although no actual calls are generated, the partial method that gets an implementation will be compiled into the assembly so that it can be invoked at runtime. The only reason for this is the Intellisense experience you get. Magic methods can be used to override default behaviors but it is necessary to expose a public method that implements the default behavior so that the user has the option to invoke this method from within the magic method if he doesn’t want to completely override it. LINQ to SQL uses this mechanism pervasively.

      Attributed methods

      Similar to magic methods, an instance or static method with the right signature can decorated with a special attribute that specifies a runtime role for this method. The runtime examines types for the presence of these attributes and registers the method to be executed on the occurrence of certain conditions. WCF Data Services uses this pattern pervasively.

      Listener interfaces

      This approach consists on defining a class that implements a custom interface provided by the framework and at a later point register an instance of this class as a listener for particular events. Usually, the interface defines one or more methods with a very specific purpose, but the listener class can be a composite of multiple interfaces. Discoverability can be improved in this programing model with a base listener interfaces and by placing all the related interfaces under the same namespace.

      IObservable<T>

      In many situations a mechanism is required to handle a stream of asynchronous events coming from the same source. IObservable<T> is an analog to IEnumerabe<T> that can be used to represent this kind of source.

      Context hooks vs. entity and property hooks

      DbContext provide a very good place for us to focus when defining extensibility hooks for anything that has to do with the functions that they encompass:

  14. Store Connection
  15. Launchpad for queries
  16. Unit of work
  17. Update adapter But for hooks that are specific to the lifecycle of entity or complex types, properties, etc., other options exist:
  18. Factor business logic into the entity type itself: The typical example of this is the OnValidate method that LINQ to SQL supports: a user can define the method on a particular entity type, and have the framework will automatically invoke the method at appropriate times.
  19. Extend the Code-First API to support hooks: The Code-First API provides a nice central configuration point for entity types, complex types and properties. We could extend the API with methods to register listeners, e.g.:

      mb.Entity<Product>().Notify(new EntityLoadedListener());
      mb.Entity<Customer>().Property(c => c.Orders).Notify(new PropertyLoadedListener());

      The one potential issue with this approach is that the hook configuration would become part of the model and therefore it would not be possible to change it once the context object has been instantiated. This might be an acceptable limitation however, since any necessary changes in behavior can be coded into the listener class itself. Making the configuration of hooks immutable also has the advantage of allowing for compiling the invocations to listeners into efficient delegates.

      EF requirements for a hooking mechanims

      We are trying to find a design that has the following characteristics:

  20. Works well for both app developers and framework developers.
  21. Makes hooks very discoverable.
  22. Invocation is efficient.
  23. Requires only very simple coding patterns to use.
  24. Should allow get the hooks at runtime and from a separate assembly.
  25. Hook handlers can be implemented outside the entity types.
  26. Hook handlers can be implemented as part of the entity types.
  27. Ideally, the way you implement a hook handler in an entity shouldn’t break POCO.
  28. The complexity of supporting multiple hook mechanism should be completely hidden from the sender. This requires further analysis, but it seems that it would be reasonable to support a mix of hook mechanisms (but not too many) to optimize for different scenarios. The current thinking is that we would use a single generic class to represent a hook. The sender should just need to call a method or instantiate a class and call a method, nothing much more complex than the typical “On[EventName]” pattern, and the hook class would make sure all the hook handlers are invoked. We are also considering building a convention system for wiring up hook handlers in the entity types. When designing this we should take advantage of any opportunity to reuse some hook mechanism as building blocks for others. For instance:
  29. If we decide that the basic mechanism is listener interfaces, then we can have context objects implement those interfaces and re-cast the hooks as events.
  30. We can have a single convention system that understands about naming patterns and attributes (similar to how the Code-First convention system understands naming patterns and data annotations as different vocabularies to express the same kind of concepts).  

      Lifecycle hooks list

      The following is an incomplete list that presents various hooks we could consider adding. Intentionally the list does not try to be specific on each hook about certain details:

  31. Hooking mechanisms
  32. Method / argument signatures
  33. All names of new hooks are up for discussion

Name	Pri	Location	Cancel or override	Description & sample scenario
QueryExecuting	0	DbContext	Yes	Query interception, custom query caching.
QueryExecuted	3	DbContext	No	When Execute happened, before the reader is read. Tracing?
QueryCompleted	3	DbContext	No	After DbDataReader is closed. Tracing?
EntityStateChanged	0	DbContext	No	Signals all state changes
EntityStateChanging	3	DbContext	?	Undo changes or change proposed values before they are set?
ConnectionProvisioning	2	DbContext	Yes	Execute additional code to make sure the connection is alive, or do logging
ConnectionReleasing	2	DbContext	Yes	Cleanup something done during Ensure / StartUsingConnection
ConnectionOpened	2		No	More likely for tracing. Since SqlClient has fixed invalid connection pools, then this is lower priority
ConnectionOpening	1	DbContext	Yes	Slightly simpler to use than Ensure/Start, would not require user to check current state. Could also be used for tracing.
ConnectionClosed	1		No
ConnectionClosing	1	DbContext	Yes	Slightly simpler to use than Release/Stop, would not require user to check the initial state. Could also be used for tracing.
OnModelCreating	0	DbContext	Yes	Tweak model before it is cached.
OnModelCreated	1	DbContext	Yes	Signal that the model is done and execute some custom code, possibly related to caching logic. . Issue: do we need this for ObjectContext? Issue: if the user is going to implement his own caching, we should have an abstract class or interface for that.
ModelCacheLookup	2	DbContext	Yes	Implement your own caching logic. Tracing?
ModelCacheHit	2	DbContext	Yes	Execute additional code when the model is found in the cache. Tracing?
EntityLoading	1	DbContext, DbEntityEntry	No	After object instance is created but before its properties are initialized. Can be used to reset a flag that will be set in newly created instances but shouldn’t be set during initialization, i.e. for validation.
EntityLoaded	0	DbContext, DbEntityEntry	No	Can be used to setup anything after an object has been materialized, i.e. event handlers, flags, etc.
CollectionLoading	1	DbContext, DbEntityEntry DbCollectiohnEntry	No	Can be used to setup anything on a collection after it is created but before it is populated. Issue: Could be used to provide your own collection?
CollectionLoading	1	Context, Entity or Collection	No	Can be used to setup anything on a collection after it has been created and populated, i.e. listeners for its changed event.
ObjectTypeResolving	1	Context	Yes	Could be used to specify a different type than the original one, i.e. to implement your own proxy mechanism. It should be per type but could return a Func<T> that returns a new instance and the result could be compiled into materialization delegates.
CollectionTypeResolving	1	Context	Yes	Something similar to ObjectTypeResolving but for collections. Could be used to replace the default collection type with a custom proxy collection with additional functionality (i.e. paging, fine grained lazy load).
Virtual OnSavingChanges	Medium	DbContext	No	Can be used to re-implement SaveChanges but still invoke the existing SavingChanges event
SavedChanges	Low	Context	No	Could be used to execute cleanup code after SaveChanges. For instance, to call AcceptChanges on each STE change tracker. It is lower priority because virtual SaveChanges covers most scenarios.
EntityStateChanging	Low	Context, Entity	Yes	For an entity instance or type in particular we could avoid putting in the modified state. So even if the properties are read-write, the context ignores changes to this entity. Could be also used to suspend fixup on an entity that is being detached.
EntityStateChanged	High	Context, Entity	No	Executes logic after an entity has been put in a certain state. Can be used to setup property values, restore state after the changing event.
PropertyChanging	Low	Context, Entity	Yes	Any time a property is about to be changed by the framework or any party, if notification or interception is enabled by the entity type. Should make original and new value available. Should also work for navigation, scalar and complex types properties. Tracing?
PropertyChanged	High	Context, Entity	No	Any time a change in a property value change is detected.
PropertyLoading	High	Context, Entity, Collection	Yes	Intercepts, overrides de loading of a property. Could be used to support loading of properties using stored procedures.
PropertyLoaded	Medium	Context, Entity, Collection	No	Tracing?
Writetable IsLoaded	High	Context, Entity	Yes	Allows cancelling the loading of a property.
CollectionChanging	Medium	Context	Yes	Any time a collection is about to be changed by the framework or any party, if interception is enabled
CollectionChanged	Medium	Context, Entity	No	Any time a change to a collection has been detected.
AssociationChanging	Low	Context, RelatedEnd	Yes	Can be used to prevent an association from being changed, or to execute business logic when the association is about to change.
AssociationChanged	Medium	Context, RelatedEnd	No	Can be used to execute additional logic after an association is changed, i.e. user can explicitly cascade relationships removals into dependent removals, workaround current databinding shortcomings.
RowValidate , RowValidateAdded, RowValidateModified, RowValidateDeleted	Medium	Context	Yes	Storage level version of ObjectValidate. Tracing?
SavingChanges event	0	DbContext	?	Currently only available on ObjectContext. Should make trigger OnSavingChanges method protected.

Existing hooks

Name	Description & sample scenario
virtual Dispose	This can be used to do additional cleanup, i.e. on entity instances.
virtual SaveChanges	Can be used to execute additional logic before, after or instead of saving changes.

Some open issues:

1. Need to prototype some coding patterns and try them.
2. Is logging and tracing part of this API? It seems that ideally we should have the same level of flexibility for hooking mechanisms with logging and tracing as we end up having with this API.
3. Second level cache should probably expose its hooks through the same mechanisms.
4. Should we provide low level query interception points with the same mechanisms, i.e. as command tress and store commands? Should we do the same for CUD store commands? Would need to make sure those work well with caching.
5. Can we get some level of support for async execution of queries with this hook?
6. Should we provide enough lifecycle hooks to implement custom fixup logic?
7. Areas of overlap with other extensibilities: read and write properties in object mapping, proxy type creation (can be imperative vs. event driven), equality and snapshot comparisons for change tracking extensibility.
8. What about customizing identity resolution?
9. Is Logging and Tracing part of the lifecycle hooks
10. Is Query interception part of the lifecycle hooks
11. Even without query interception we should expose when we are about to execute (imagine a profiling tool that measures how much query compilation costs).
12. Is ContinueOnConflict part of the lifecycle hooks
13. How do we improve diagnostics? Can we have OnError
14. Need to do prioritization, costing and scoping
15. Should we have a fine grained version of CollectionAdding / CollectionRemoving with support for magic methods on the entities to enable collection patterns? We would need a pattern for Contains checks also.
16. There is a conversation about splitting AssociationChanged this event into properties and collection changes. However, there should be a way to tell the difference between a scalar property change and a nav prop. Should we make AssociationChanged more accessible and add AssociationChanging? This would provide a way to intercept changes in associations independently of cardinality, constraints, etc.
17. AssociationChanging would need the entity and collection types to collaborate to avoid changes from being made to the graph.

[natemcmaster]

I started prototyping some ideas as a part of my investigation of setting SQLite pragma options when a connection opens. It seems this feature would be generally useful in solving several other feature requests. Can we considering moving this off backlog?

[rowanmiller]

I think we need @divega in the office to design this one, the all up feature is pretty large and wide-reaching. We also have some higher priority things to work on first. If you are specifically looking at connection pre-amble then I think we can treat that as a smaller feature that we work on now.

[janhartmann]

Will this make it into the next release?

[Antaris]

@rowanmiller In terms of an EF Core ObjectMaterialized style hook, can you give me some pointers as to where I can tackle this as of now?

[divega]

Permission validation and filtering are other scenarios that could be attained with lifecycle hooks (see #6440).

[dazinator]

Will this also take care of the following scenario?

I'd like to add an entity to a DbContext (to be inserted or updated etc), but also subscribe to be notified once the entity has been saved, so i can handle taking its newly updated values (i.e the database generated values that get poplated after a SaveChanges() - like it's ID etc) and do something with them.

                var dbContext = GetExistingDbContextInstance();           
                var newItem = new SomeEntity();
                dbContext.SomeEntities.Add(newItem);
               // I am quite far down in an object graph, and SaveChanges() will eventually be called
               // later on by something higher up, co-ordinating this transaction. I'd like to be notified
              // here though once SaveChanges() has been called and this entity has been persisted, so I can grab the
              // the updated entity values.
              //   dbContext.OnceSaved(newItem, ()=>{   // ooh someEntity.Id is now populated! })  

Something along the lines of:

 dbContext.OnceSaved(someEntity, ()=>{   // ooh someEntity.Id is now populated! })

Where the callback would be invoked after dbContext.SaveChanges() is called.

[Antaris]

@dazinator I'm currently working on an extension project for EFCore (https://github.com/Antaris/EntityFrameworkCoreExtensions) which will allow you to do what you want, eventually.

I have a number of working hooks including change tracking, value materialization, querying and storage. It's still a WIP though.

[SidShetye]

@rowanmiller What's the expected release date on this? It's been open for almost 2-1/2 years

[rowanmiller]

@sidshetye it's planned to be in our 2.0 release (at least the first set of hooks). No specific dates on that yet, but it's the release we are starting work on now.

[tpanthier]

In the mean time, using EF Core 1.1, what would be the recommended method for executing a SQL statement straight after a DbConnection is opened?

We are using RLS for implementing multitenancy and I'd like to know if there's any method for setting the TenantId until EF Core 2.0 is released.

[ajcvickers]

@tpanthier You could try registering for an event on the DbConnection itself.

[ajcvickers]

Punting this because we are out of time. /cc @divega

[michaelaird]

With this issue being open for 2.5 years and the various other issues that have been closed due to referencing this issue, I've lost track of whether two specific scenarios we care about are covered by other cases and are or are not supported in 2.0.

1. As part of our multi-tenancy solution, we currently subscribe to DbContext.Database.Connection.StateChange. Whenever the state changes to Open, we issue a SQL command to set the the current tenant id in session_context.

2. In EF 6, dates loaded from the DB have a kind of Unspecified. In order to coerce these into UTC, we're hooking into ((IObjectContextAdapter)DbContext).ObjectContext.ObjectMaterialized and calling DateTime.SpecifyKind for specific fields that we care about being in UTC.

Can you confirm if either or both of these scenarios will be possible with EFCore 2.0?

[ajcvickers]

@michaelaird Unfortunately, there is no change on either of those issues for 2.0. There is a high chance that they will make it in for 2.1.

[michaelaird]

Got it. Thanks for the update.

Off-topic but slightly related: My observation of the 2.0 milestone across all the .net core projects is that it represents a solid "step up" in stability and general breadth of feature support. With that stability in mind, it would be great if there were more frequent pre-release packages available between 2.0 and 2.1. In particular it would help us get to try these features that just barely didn't make the 2.0 release....

[ajcvickers]

@michaelaird I hear you. On the EF team we don't really control when the releases happen or how many pre-releases there are, but we do push for them to be frequent as much as we can.

[SidShetye]

Guys - do you genuinely think it's acceptable performance when you've had 2.5 years to track, plan and execute - and yet it consistently misses releases? This is open since Sept of 2014. Very disappointing. Especially knowing you guys are clever :(

[michaelaird]

hey @sidshetye , I don't know if you ship commercial software for a living, but I do. And I aim to get things into releases all the time that get re-prioritized by the business and other things that are more important get pulled in ahead. It drives me crazy, as I'm sure it does to the EF team, when I don't have enough hours in the day to deliver all the features that I know my users want.

Let's support the EF team in what they are delivering and not beat them up too much for what is missing the cut.

[dazinator]

@michaelaird I agree, however negative feedback is as important as positive feedback and both kinds are better than no feedback at all. If you aren't getting any feedback it usually means no one cares! What you are witnessing is someone who cares enough about EF to express their dissapointment over an issue and I think it's perfectly valid and even useful for the EF team to hear that. Any attempt to silence such a thing would not be constructive imho. I am also certain that the EF team, being highly professional, are perfectly capable of handling both positive and negative feedback on issues without needing any one of us to jump in for them.

[SidShetye]

@michaelaird Yes, I ship commercially. Including security and voice for 60% of all mobile phones - globally. Code that defined "carrier grade". But making this about me is a distraction - the observation stands even if I personally never shipped a single line of code.

Since we're quickly off topic now, I'll get back with the following suggestions to the EF team:

1. If I've offended anyone on the EF team, please accept my apologies since the goal was never to upset anyone but to draw attention that this is a serious for some folks. Without this, EF 7 isn't a shipping product. And .NET core isn't a production option (for us and customers). Because ... dependencies.
2. Please, please spend more time on customer communications. While intricate foreign policies and constitutional debates can be done via 140 characters, customer communications (unfortunately) needs more.
3. For this specific issue, given the delays - could you at least share the final expected interface/design? I understand ironing out the underlying implementation may take more time. Could be a simple mapping of EF6 hooks (e.g. ObjectContext.ObjectMaterialized and ObjectContext.ObjectContext) to their equivalent EF7 hooks + newer hooks.

I hope my/anyone's feedback doesn't distract from the teams achievements. At the end of the day, we're all trying to make the world a better place through better software. Just don't keep us waiting ;) !

Thanks, Sid

---

This turned out longer than I expected, so I'll get back to work. Cheers.

[ilya-chumakov]

As we don't have hooks yet, I would like to ask about DbContext.Database.Connection.StateChange event. Is it safe to subscribe on it without un-subscribe? What is DbConnection CLR object lifetime if EFCore is used in ASP.NET app? As I see, IRelationalConnection is registered as Scoped in DI, and DbConnection's lifetime is the same. I could assume if the subscriber also is registered as Scoped, we could relax and don't worry about memory leaking... or not?

@michaelaird, @ajcvickers could you help?

[ajcvickers]

@ilya-chumakov When EF creates the DbConnection for you, then it's lifetime is the same as the DbContext--it will be disposed when the DbContext is disposed. You can also create your own DbConnection and pass it to UseSqlServer (or equivalent) in which case you can make the lifetime longer.

So, if your subscriber has the same lifetime as your DbContext and EF is creating the DbConnection, then the subscriber, context, and connection should all have the same lifetime.

[sjb-sjb]

@rowanmiller commented that some hooks are coming in 2.0, and in https://github.com/aspnet/EntityFramework/wiki/Roadmap there is a one-sentence description, but I haven't been able to find anything else on it. Is there more documentation or pointers out there?

I am interested in hooks for entity state transition. To this end I am using the following approach:

        var localViewListener = dbContext.GetService<Microsoft.EntityFrameworkCore.ChangeTracking.Internal.ILocalViewListener>();
        localViewListener.RegisterView(StateManagerChangedHandler);

However the callback for entity state transition does not provide all the information one might desire. Specifically, for example, if a transition from Modified to Unmodified occurs then we do not know whether this is due to AcceptAllChanges or due to some form of Reject Changes, or possibly due to some other reason. For that, you need to add a flag in your calling code. You need this information to know, for example, in your entity view model whether the current value has changed or the original value has changed. So when you get around to the hooks for entity state transitions, I would ask that you include some indication of whether AcceptAllChanges is in progress or RejectAllChanges is in progress (and I'm hoping you will add RejectAllChanges !)

[divega]

@sjb-sjb if you look in the roadmap you will see that lifecycle-hooks is in the list of "Features originally considered but for which we have made no progress and are essentially postponed".

I will let @ajcvickers comment on whether there is any reasonable way to workaround current limitations and get notified of those changes by using internal APIs but without having to wrap the whole set of change tracking services, but that seems unlikely.

[ajcvickers]

I can't think of a workaround off the top of my head.

[kdcarlisle42]

Are there any, or going to be any, workaround/solutions for DbInterception? Based on this thread and the roadmap it seems that it's not going to be available for awhile. We are starting some new projects and want to use .Net Core / EF Core, but I'm not sure how to translate our existing DbInterception process.

We are using interception to inspect the sql query and replace [SECURITY_VIEW] with an actual view name. In our current system, each user has their own security view. Views could be added/removed each day, so we cannot create a static context.

In our latest code, which uses EF6, we tested dynamically compiling the model vs interception. Once we started load testing, we found interception to be the winner.

We could go back to writing sql stored procedures, but I'd prefer not to.

Any thoughts/ideas would be greatly appreciated.

[ajcvickers]

@kdcarlisle42 You could investigate using the DiagnosticSource events documented in https://github.com/aspnet/EntityFramework/blob/dev/src/EFCore.Relational/Diagnostics/RelationalEventId.cs

These events give you access to the DbCommand, etc, before it is executed, and it may be possible for code to manipulate these objects to do what you need. However, it's not as flexible as DbInterception in EF6. I would be interested in hearing about your experience if you go this route.

(Note that these events have changed significantly between 1.1 and 2.0, so if you write something for 1.1, then it will need to be modified to work with 2.0.)

[sjb-sjb]

I don't know what the limitations are that were referred to by @divega and @ajcvickers. Could one not simply add an event like this to DbContext?

    public event TypedEventHandler<IEntityStateWatcher, EntityStateTransitionEventArgs> EntityStateTransition;

    public DbContext( ... )
    {
        ...
        var localViewListener = this.GetService<Microsoft.EntityFrameworkCore.ChangeTracking.Internal.ILocalViewListener>();
        localViewListener.RegisterView(StateManagerChangedHandler);
        ...
    }

    private void StateManagerChangedHandler(Microsoft.EntityFrameworkCore.ChangeTracking.Internal.InternalEntityEntry entry, EntityState previousState)
    {
        var entityEntry = entry.ToEntityEntry();
        this.EntityStateTransition?.Invoke(this, new EntityStateTransitionEventArgs(entityEntry, previousState));
    }

[olman21]

I've been following this issue for a couple of months, I need to perform a soft-delete feature in my Application, I know how to "override" actual deletes with "update the delete flag" but not a clue how to filter the "soft-deleted" records once a query is performed, this would be easy with hooks, but in this case I can't imagine how to workaround this.

[ajcvickers]

@olman21 Take a look at Global Query Filters: https://blogs.msdn.microsoft.com/dotnet/2017/05/12/announcing-ef-core-2-0-preview-1/

[ajcvickers]

Consider what can be done to observe transactions commit/rollback--see #9429

[KallDrexx]

Can this task (if it ever gets prioritized) support passing some sort of timings in with the hooks?

Right now our Asp.net core with EF core running on Azure is having periods of non-reproducible high 75, 90, 95, and 99 percentile response times. We want to hook into DB operations so when our special request logging is returned it returns the # of queries and average db request/response time for each so we can see if this inconsistency is db related or not.

Right now it seems possible to do this for SaveAsync and FindAsync calls on the DbContext by overriding those calls and putting a stopwatch around them, but I don't see an easy way to get the timings of any Linq based queries, without manually adding code around every single EF core code block.

[SidShetye]

Still looking for ObjectMaterialized and SavingChanges equivalent hooks. With the passage of time (and .NET standard 2.0 out), do we have a cleared release date? Pretty please?

[divega]

@sidshetye In case you haven't noticed this issue has been recently assigned to the 2.1 release as a "stretch priority". The exact hooks we will prioritize hasn't been decided yet, but your input will be considered.

[divega]

@KallDrexx it sounds like you should be able to use the logging & diagnostics functionality in EF Core for this. Have you had a chance to try that?

cc @ajcvickers

[KallDrexx]

@divega Maybe I am missing something but I can't find any documentation that shows how this can be effectively used, at least for my use case.

The first issue is I don't see much in the logging facilities that would allow me to output query times (including network times) so I am not totally sure how to grab that.

The second issue is that in production we whitelist what logs are produced to keep costs down and to keep logs concise and easy to search through (i.e. one log record per request with all relevant data in a json record). This makes it extremely easy to utilize tools such as Elastic Search to find complete details about a total request instead of making hard assumptions and having to coalesce 20 different log entries to get a complete view of a single request.

From what I can currently see, utilizing the current logging infrastructure means creating a custom ILogger with a lot of non-trivial hoop jumping to get query data to flow out of our custom Logger into the request context, so at the end of the request the data can be coalesced into our simple request.

[divega]

@KallDrexx if logging doesn't meet your requirements, EF Core also publishes diagnostics events using DiagnosticSource.

There is some information about how to consume this information programmatically at https://github.com/dotnet/corefx/blob/master/src/System.Diagnostics.DiagnosticSource/src/DiagnosticSourceUsersGuide.md#consuming-data-with-diagnosticlistener. It also possible to consume this diagnostics events through ETW on Windows.

[KallDrexx]

Thank you very much, I had not known that there was a diagnostic system built into .net core, so I will investigate that route. Really appreciate the response.

On Mon, Aug 28, 2017 at 1:31 PM, Diego Vega notifications@github.com wrote:

@KallDrexx https://github.com/kalldrexx if logging doesn't meet your requirements, EF Core also publishes diagnostics events using DiagnosticSource.

There is some information about how to consume this information programmatically at https://github.com/dotnet/ corefx/blob/master/src/System.Diagnostics.DiagnosticSource/src/ DiagnosticSourceUsersGuide.md#consuming-data-with-diagnosticlistener. It also possible to consume this diagnostics events through ETW https://github.com/dotnet/corefx/blob/master/src/System.Diagnostics.DiagnosticSource/src/DiagnosticSourceUsersGuide.md#consuming-diagnosticsource-data-with-eventlisteners-and-etw on Windows.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/aspnet/EntityFrameworkCore/issues/626#issuecomment-325420694, or mute the thread https://github.com/notifications/unsubscribe-auth/AAIggYAUt_9e-jdhU8-9EajopCLRy0nWks5scvlugaJpZM4Cesel .

[SidShetye]

@divega thanks for the update, missed that tag update. Though it's not very comforting that this is as low of a priority as 'stretch' / "if we have bonus time left".

To us this is a deal breaker for EF core (and therefore .NET core). Hope .NET core and EF core get more investments. It ultimately boosts Azure deployments - at least for us.

[mguinness]

@sidshetye Although not ideal, a couple of possible workarounds for your needs are a custom Entity Materializer Source and another library EntityFramework.Triggers.

[AndriySvyryd]

@ajcvickers Add hooks for customizing a model and the model cache key using with a context instance

[codetuner]

For a concrete application of SavingChanged and ObjectMaterialized in EF6, see my article about hybrid storage on CodeProject. Inspired by the Core way of doing things, I defined an extension method "UseHybridStorage()" to DbContext that registers to the ObjectMaterialized and SavingChanges events. Hence no subclassing is needed. It would also be very usefull to have access to the Context from within those lifecycle hooks.

Would what I did in the article be possible with EF Core 2.1 too ? (See the OnObjectMaterialized and OnSavingChanges methods just above the "Entity State Handling Issue" paragraph.)

[jaredcnance]

I think it would also be beneficial to have some form of hook to notify if a LINQ expression was not able to be translated. I am aware that this gets logged, but it would be helpful to have a programmatic way to escalate it from a simple WARN log to some other form of notification. This, of course, makes the most sense during development but it is not always obvious to the developers.

The primary situation I'm thinking of is one where a library (for json:api or graphql as examples) sits on top of entity framework and is responsible for interacting with EF. This would enable the library to surface these kinds of problems in a more obvious way to the developers and provide framework-specific solutions.

[ajcvickers]

@janhartmann It is already possible to:

• Configure this as an error instead of a warning--if catching an exception in this case is acceptable.
• Hook into the event (using either ILogger or DiagnosticListener) and do whatever with it. For example, using DiagnosticListener:

public class MyListener : IObserver<DiagnosticListener>, IObserver<KeyValuePair<string, object>>
{
    public void OnCompleted()
    {
    }

    public void OnError(Exception error)
    {
    }

    public void OnNext(DiagnosticListener listener)
    {
        if (listener.Name == DbLoggerCategory.Name)
        {
            listener.Subscribe(this);
        }
    }

    public void OnNext(KeyValuePair<string, object> value)
    {
        if (value.Key == RelationalEventId.QueryClientEvaluationWarning.Name)
        {
            var payload = (QueryModelClientEvalEventData)value.Value;
            Console.WriteLine("Client eval happening: " + payload);
        }
    }
}

Then somewhere in app startup:

DiagnosticListener.AllListeners.Subscribe(new MyListener());

[ErikEJ]

This would be nice to have in docs

[ajcvickers]

Thanks Erik. Filed https://github.com/aspnet/EntityFramework.Docs/issues/583

[SidShetye]

@ajcvickers @divega : is there a firm release date we can go by? I have multiple customers asking us to expand Crypteron to support EF Core. We've been deferring them to EF6 for years now but I'm wondering if 2017 will be special

[sjb-sjb]

I would like to renew my vote for state transitions to become part of the public API (see Jul 13 above).

[ajcvickers]

@sidshetye Unfortunately we don't have any dates that we can share yet.

[SidShetye]

@ajcvickers That's disappointing. It makes roadmap planning impossible for your customers (us and in-turn our customers). We certainly appreciate the work by the team and I know it's comforting to not state a release date (that you're now held accountable to!). But may I challenge the team to step outside that comfort zone? There has got to be some plan and some target date ... doesn't help to keep the community in the dark.

Again, love the work - just wish there was more transparency to facilitate business.