[Proof of Concept] Faster database queries and less boilerplate with automatic entity navigation

[zaneclaes]

Motivation

As I went through the tutorials, I couldn't help but notice some inefficiencies in the DataLoader implementation(s), which are over-fetching by an order of magnitude or so. It also seemed like there was an opportunity to reduce boilerplate code.

I have created a working, generic solution to this problem using EFCore5's navigation properties.

Improvements

To take the first DataLoader example (Speakers -> Sessions)...

• The example requires manually defining SessionSpeakers, which loses the benefit of EFCore5's implicit many-to-many feature.
• The SpeakerResolvers is inefficient. It starts by re-fetching a well-known Speaker (which should already be present from the top level query), and then joining on the intermediate table. Only then does it enqueue the ID of the Session which is to be loaded by the Loader. Therefore, a GraphQL which returns 10 speakers will actually require 12 queries (1x Speaker + 10x SpeakerSession + 1x Session).

This latter point is a classic N+1 query. Avoiding N+1 queries is pretty much the whole point of using a data loader.

Syntax (Example)

With this proof-of-concept, it is assumed your models follow standard entity relationships. I have focused on many-to-many relationships in EFCore5 with implicit tables to demonstrate this feature.

First, for convenience, create a concrete extension for your application primary key type and DbContext type (string and OWSData in my case):

namespace OpenWorkShop.Data {
  public static class OWSEntityResolverExtensions {
    public static IObjectFieldDescriptor Entity<TData, TProp>(
      this IObjectTypeDescriptor<TData> desc, Expression<Func<TData, ICollection<TProp>>> propertyOrMethod
    ) where TData : IModelId<string> where TProp : class, IModelId<string> =>
      desc.Entity<string, TData, TProp, OWSData>(propertyOrMethod);
  }
}

Then, to set up the many-to-many relationship, just use:

descriptor.Entity(t => t.Sessions).Name("sessions");

That's it.

You delete any code for:

• SpeakerSession
• SpeakerResolvers
• SessionByIdDataLoader

Under the Hood

Instead of implementing the Resolver by querying the Speaker table for a second time (and including the SessionSpeakers), this extension insects the navigation properties on both classes. Then, it directly queries the intermediate table (without any joins) to discover the involved sessions. Finally, it queries the sessions table to return the results.

The total number of database queries should always equal the number of tables involved (four, in this case).

Testing

I'm using this with 6 different classes, each of which have a variety of many-to-many relationship. With the extension set up, it's just one line of code per relationship (descriptor.Entity()). So far, it's all working flawlessly.

n.b. I ported this from work I did with the GraphQL library, so I had already tested a fair bit of the code.

Limitations

• As I said: I've focused on Many to Many (only).
• Your models must implement IModelId<TKey> — I simply haven't bothered to intergrate with AsNode().IdField(...) in order to infer the Id of your model. Instead, this proof of concept expects your models to implement the TKey Id { get; } property. This limitation could be removed with further development.
• Your models must be well-formed, with exactly one ICollection<OtherType> property available, per the many-to-many convention.

Source Code

https://github.com/zaneclaes/hot-chocolate-entity-framework-resolver

[michaelstaib]

If you do an initial pr I can add compiler support and attribute support so that it works with fluent but also with attributes.

[michaelstaib]

When you clone the repo, run:

./build.sh restore

Which will restore the packages and generate a new solution called All.sln located in src.

If you are using vscode like myself, you can also use the scoped solutions... so basically, head over to data and use the HotChocolate.Data.sln instead.

EF specific code goes here: https://github.com/ChilliCream/hotchocolate/tree/develop/src/HotChocolate/Data/src/EntityFramework

[zaneclaes]

Great — CI tool ran, and project is building. Took a while because I'm on DSL in rural Colorado ;)

[zaneclaes]

@michaelstaib do you have a recommendation how to ditch the IModelId<TKey> requirement? I think what I want is a way to access the INodeDescriptor<TNode>. Given only the TNode, I need a way to determine the TId, as well as retrieve its value from a TNode instance.

[michaelstaib]

Yes, with a lazy config .... we will hook in and wait until the init phase is completed and then grab the node member. But we could even solve this more generically by resolving the Key member whenever we do not find node.

[zaneclaes]

Is there an existing mechanism I can call so that this is included in my PR, or is this something you intend to build?

[michaelstaib]

All is in place ...

There are various ways to do this:

• Extend()
• LazyConfiguration
• TypeInterceptor

So, the easiest way to I think is like the following:

descriptor.Extend().OnBeforeNaming((c, d) =>
{
    ....
})

The type system is initialized in three phases....

1. everything is being registered.
2. types are being named
3. types are being completed.

So,

we will hook in just right before 2 and then just grab the member from the node field. If the type has no node field we will just look the key prop up.

d.Fields.TryGet("id" ....

We could tell the node descriptor to put a node member on the context to make this even easier....

can you create a PR and I can put some stubs in for this?

[zaneclaes]

Sure, if that's easier for you.

As I get it up, one other related question:

What is the idiomatic HotChocolate way to access the user-configured/scoped DbContext instance (should I just GetRequiredService<DbContext> or otherwise DI the generic type)? I don't want to make the user re-type the DB context; it inflates the number of generics required to be passed.

[michaelstaib]

The best way to access the dbcontext is through the new IDbContextFactory. This allows the execution engine to fetch in parallel. On the configuration side, it should be configured to use pooling so that we do not have too many allocations around creating it and dropping it.

[zaneclaes]

Yeah, I'm using that, but what I'm trying to ask is about getting rid of the <TDbContext> requirement. It seems IDbContextFactory requires typing, which I do not wish to provide — so as to avoid additional generic typing on the behalf of the end-user.

Specifically, IDbContextFactory<DbContext> fails to resolve.

[michaelstaib]

I will have a look at your code again later ... but yes there are ways to store it on the context ... we do that with a lot of other things.

[nguyenquyhy]

Is there any update on this or any new recommendation on using Entity Framework? I can't find much information on using HotChocolate with Entity Framework in the v11 docs.

Right now any navigation properties of Entity Framework will return null, or Cannot access a disposed context instance. if lazy proxies is used. So I have to write individual field descriptor for each navigation to overwrite EF navigation property. Am I doing it wrong and is there a better way to use EF than this?

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.