How to Express Model Metadata on the Client

[weitzhandler]

In the previous version of TE I struggled a lot with having the client to take care of foreign keys and references of all variations (1-M, 1-1 etc.).

It would be a good idea to have a way to let the client know in foreign keys, what entity is the parent and what's the child, and whether there is a dependency.

Imagine this scenario:

public class Runsheet
{
    // 1..* towards Responder, using RunsheetResponder as a bridge table
    public virtual ICollection<RunsheetResponder> Responders { get; set; }
    public Child Child { get; set; }    
}

public class Responder
{
}

public class RunsheetResponder
{
  public int RunsheetId { get; set; }
  public int ResponderId { get; set; }

}

public class Child
{
  // 1-0..1 to Runsheet
  [Key, ForeignKey]
  public RunsheetId { get; set; }

  // ****** This is what my post is about ***** //
  // When I set this property to an existing RunsheetResponder in the client-side, or to null,
  // I want to be able to configure TE
  // whether it should mark the attached entity as `Added`/`Deleted` respectively,
  // or it should just instead only set the RunsheetResponderId to null,
  // while leaving the state of the attached/detached intact.
  [ForeignKey(nameof(RunsheetResponderId))]
  public RunsheetResponder AttachedBy { get; set; }
  public int? RunsheetResponderId { get; set; }
}

This is one of many scenarios I bumped into when using TE 1, and had a rough time addressing.

The idea is, that to avoid DRY code everywhere in the client/server side we need to have a means of configuring the references and relationships ownership and 'cascadance' transferred to and analyzed by the client.

The goal is to automate all the relationship and foreign key properties management, especially in the client-side. In cases where manual control is needed, this should be done by either adding attributes on the relationship/foreign-key properties in the server-side, or by injecting an implemented interface that provides information about the relationship ownership configuration (for example IConfiguration<TParentEntity>).

Does this make sense?

[tonysneed]

I think it's a going to be necessary to have model metadata on the client in order to do change tracking properly. In classic TE this is done via code generation, for example, by adding a Parent property that the change tracker uses to handle many-to-many relationships. But that approach has gaps, as you pointed out.

So what we need is something akin to what Breeze does to transmit model metadata. The issue, however, is that their toolchain relies on CSDL, which is not going to be part of EF Core. But the OData team should have a replacement, because they need something to replace CSDL. They are now in beta, so we'll need to post an issue there to ask about the metadata.

All that said, this will require a re-write of the client-side change tracker for .NET Standard. I'm going to need help with implementing that. The TypeScript client trackable entities is going to also need model metadata.

[weitzhandler]

I think it's a going to be necessary to have model metadata...

Even if we keep using client entity generation (the dev might want to choose entity sharing or other methods), we still want to supply the generated data in an extensible way, so some configurations not disclosed by EF should be configurable by implementing interfaces or decorating the appropriate entities with attributes.

But the OData team should have a replacement

I just discovered the the OData.NET is alive, as opposed to me thinking it was dead. I hope support for .NET Core 2.0 and EdmLib support for EF Core is implemented or is in advanced stages.

I'm going to need help with implementing that.

I'll be happy to take assignments.

The TypeScript client trackable entities is going to also need model metadata.

1. Bare in mind that BreezeJS is working on this at the moment.
2. I don't speak TypeScript fluently and in fact haven't used it a lot (I have been developing only desktop/mobile client apps that is WPF/Xamarin), and although I'm willing to improve my TS, I'm not sure I can take serious tasks for the moment.

[tonysneed]

Can you work up a proof-if-concept using Data Annotation attributes? It’s Bear if we can leverage those rather than inventing our own.

[weitzhandler]

Can you please give an example to what DA attributes you mean, and how it can help us determine the model dependencies structure?

[tonysneed]

In your initial question above, you provide an example of related entities where you apply DA attributes to specify the model relationships -- [Key] and [ForeignKey] I believe what you're asking for is a hook in the client-side change tracker ChangeTrackingCollection<T> which allow you to override the default behavior and mark the TrackingState of the entity as you see fit. Is that correct?

[weitzhandler]

Even tho I used attributes in my example, learning about the server-side model should be only done via EF. Besides, I dug a bit more into EF Core and I saw that it supports things that wasn't previously out the box, for instance cascaded deletes, that can now be configured very easily and efficiently with the fluent API.

It would be nice, although not critical, to have open extension points, where we can customize and add interceptors in both the server and client side functionality. For example, instead of using extension methods for the ChangeTrackingCollection<>, we can have make the ChangeTrackingCollection have a constructor that accepts a IChangeTrackingCollectionManager etc. that can be overridden and configured or provided via IoC. The subsequent CTEs will initialize using the parent manager, something like that.

Also, (now that .NET Standard is out), maybe we would want to consider using some code proxying library such as trackerdog that auto-implements change-tracking, or code weaving library such as Fody, to have all the tracking code generated for us in the house.

[tonysneed]

Even tho I used attributes in my example, learning about the server-side model should be only done via EF. Besides, I dug a bit more into EF Core and I saw that it supports things that wasn't previously out the box, for instance cascaded deletes, that can now be configured very easily and efficiently with the fluent API.

Yes, the model can be specified either using attributes or via a fluent API. But in EF Core we lose the ability to express the model relations using XML or JSON (in EF 6 this is done via CSDL). So the only choice we have on the client would be to use attributes, because the client is ignorant of EF but can still use the Data Annotation attributes, which are neutral regarding an specific data access stack.

It would be nice, although not critical, to have open extension points, where we can customize and add interceptors in both the server and client side functionality.

TE already has an interception mechanism for the server, so that you can override the behavior of ApplyChanges and add logic there. What I'm thinking is to add something on the client that lets you override setting TrackingState, perhaps a callback. I'll think about the IoC approach and what makes sense here.

Also, (now that .NET Standard is out), maybe we would want to consider using some code proxying library such as trackerdog that auto-implements change-tracking, or code weaving library such as Fody, to have all the tracking code generated for us in the house.

I'm not familiar wit these libraries, but I'll have a look.

[weitzhandler]

Yes, the model can be specified either using attributes or via a fluent API. But in EF Core we lose the ability to express the model relations using XML or JSON (in EF 6 this is done via CSDL). So the only choice we have on the client would be to use attributes, because the client is ignorant of EF but can still use the Data Annotation attributes, which are neutral regarding an specific data access stack.

In the other hand, EF Core can collect more data about the model (especially about cascades and dependencies) than EF could. I'm sure we can transfer there is a way to pass this information to the client. Be it in any possible means. Attributes just doesn't seem right, that's how I see it.

TE already has an interception mechanism for the server, so that you can override the behavior of ApplyChanges and add logic there. What I'm thinking is to add something on the client that lets you override setting TrackingState, perhaps a callback. I'll think about the IoC approach and what makes sense here.

I think we should replace the extension methods with IoC controlled classes that can be overridden. Imagine the ChangeTrackingCollectionExtension class would be a virtual class(es) provided to the ChangeTrackingCollection graph-root (optionally via IoC). This will give much greater control in what we want to override, given we can divide up the main methods (for instance AcceptChanges) into smaller ones, with some of them made virtual, and will allow adding more interception methods, like OnChangesDetected, OnGraphChanged etc. etc. So we could have class ChangesDetectorBase : IChangesDetector which has a constructor accepting a change tracking collection which can be provided by IoC (or will use the default one), and some other classes each doing a separate job and available for overriding and re-registration with the injection container. Same applies in the server side's ApplyChanges and LoadRelatedEntities, we should have a registered class like ApplyChangesService that offers a default implementation chunked up into bits of virtual methods, and is provided to the DbContext or to the controller via dependency injection. I mean ASP.NET Core is all driven by IoC nowadays.

[tonysneed]

I'm a big fan of IoC as an extensibility mechanism. That way, anyone can register an alternative implementation if they want to.

So maybe let's start with adding an IChangeTracker interface to the ctor of ChangeTrackingCollection in the TE client package (which will be ported to .NET Standard), and the job of this guy would be to set entity TrackingState with a SetState method. The implementation could use reflection to inspect model metadata attributes, for example. I agree that this approach based on attributes is limited, but the beauty of IoC is that it could be replaced by a future implementation that uses rich model metadata, if and when a format becomes available on the EF Core side. ( I asked the EF team about it, and they have no plans to express model metadata in a format that can be consumed by a client.)

[weitzhandler]

I haven't done serious plumbing with the EF Core yet, is there ANY way to extract model metadata from the DbContext? How to serialize this info to the client we'll already find a way, but is there a way to extract it from the EF model at all? We might have to create a compact Edm library (shared) to be able to build up the model descriptors and its metadata to be transferred and analyzed to the client, we have to strain towards getting the MD from EF rather than extracting it ourselves. If there isn't a proper transmission way, we might want to use an external library (EdmLib?), or make up our own. I'm not sure we need to start with all the possible complexities, but these features should be covered:

• All relationship types (1-1, 1-M, ~M-M~)
• Relationship dependencies (principal vs. dependent)
• Non-entity types (such as owned types)
• Required references (for example 1-0..1 vs 1-1)

[tonysneed]

That’s a lot of work, which may already be done by someone else. I’ll open an issue on the OData for .NET Core repo to see what they have planned to express model metadata.

[weitzhandler]

I hope they already have a mapper between EF Core and EdmLib.

[tonysneed]

The .NET Core version of OData is on the feature/netcore branch of the OData / WebApi repo. There is a MyGet feed for the nightlies. And I've opened an issue with the question about the $metadata endpoint.

[tonysneed]

I'm doing a proof-of-concept today on OData for ASP.NET Core, and I'll see if a metadata endpoint is exposed by default.

[tonysneed]

Let's schedule a Skype chat on Sunday to discuss how to approach the issue or model metadata on the client. I am actively working on the EF Core scaffolding toolchain, and I have a good idea how we can leverage that to express the model metadata in a way that is not coupled to EF itself and therefore can be used on a client that is ignorant of the server-side persistence stack (EF, No SQL, etc). The key here is to use a client-side API that is not EF-specific. All we want to do is express the model relations so that a client-side change tracker can reason about them in order to set TrackingState when some property is changed or entities are add / removed from a related collection.

[weitzhandler]

Probably important: ASP.NET Core OData now Available.

[tonysneed]

Yes I heard, but last time I checked I don’t think it has the concept of metadata, because EF (Core) no longer uses CSDL, which OData projected as metadata.

[weitzhandler]

What do you mean? Isn't there anywhere we can get internal data (such as field size or restrictions) on the models? What about this?

[tonysneed]

Ok it looks like they added since beta.

[tonysneed]

Deferring this until there is more usage of TE by the community. @weitzhandler Appreciate your ideas!