Proposal: Break out OData Parsing and decouple from EF Core

jhancock-taxa commented 2 weeks ago

Right now, OData is heavily dependent upon Entity Framework and also Controllers.

In an effort to bring OData to minimal apis and even FastEndpoints and expand out the offering so that it can work in more places (i.e. NoSQL Databases like RavenDb and Mongo without EF) I think it would be beneficial to rethink a little bit about what is going on and why:

There are 7 things that OData is commanding when effectively generating an SQL Query but with Linq and Expression Trees:

Where - The filter on the data that will occur.
Order By - The sorting.
Group By - How the data is being grouped.
Expansions - Define what values should be "included".
Select - What shape should be returned.
Pagination - Total records available and the slice that should be returned from those total records.
Aggregation - Aggregates using the extensions: https://devblogs.microsoft.com/odata/aggregation-extensions-in-odata-asp-net-core/

It should be possible to Pass all of this in with DI into any Minimal API or even Controller or Fast Endpoint as a scope object that gets the HttpRequest and parses the values. If it is a generic injection with the type of the data object that is being operated on (i.e. your DTO) then the expressions and values for the above can all be strongly typed.

Consider the following DI injected Object:

{
   Expression<Func<T, bool>>? Where {get; init;}
   Expression<Func<T...>? OrderBy {get; int; }
   GroupBy,
   Select,
   Expand,
   aggregates,
   long? Top,
   long? Skip,
   bool IncludeCount
}

If this was passed into your MinimalApi or even Controller, you could build up what you need yourself. And, you could have the ability to create an extension method in OData that takes an IQueryable and allows ApplyOData() which returns the result with everything just done for you. You could also have extension methods for every step above for ApplyWhere, ApplyOrderBy etc.

This would be the simplistic case that would be a perfect match for what OData already does but updated for .net 8 and the new realities and would eliminate some of the magic while vastly expanding the use cases.

Then there exists an opportunity to provide the tooling to fix the OData Select Before Where problem. That is, if you have a DTO that OData is operating on and it isn't the Data Class (which it shouldn't be but OData currently forces) then you need to have something that convert the expressions for Where, OrderBy, GroupBy, Select, Expand from the DTO to the Data Object. Tools like AutoMapper have AsDataSource() and Mapperly is working on something similar (https://github.com/riok/mapperly/issues/293) that would enable taking the Expression<Func<TDto,bool>> Where of the above as an example and map it to Expression<Func<TData, bool>> by reversing the mapping from the TDto to the TDataObject and thus allow OData to operate against DTOs and then be translated into something that can be properly executed against the database, which also has the side effect of completely decoupling OData from the data model.

This allows every one of the expressions that OData is creating to be translated so that the Where can happen before select and have the right field names etc.

While I'm aware that EF Core linking allows OData to get around things that Predicate builder handles, I think that it's entirely reasonable to use Predicate Builder to do this for OData and simplify down this entire process and make it vastly less opaque and in the process vastly more usable in a vastly more use cases.

This should also enable making public the code that allows for example taking the $filter and parsing it into Expression<Func<T, bool>> separately as a static function or an instantiated class if necessary that outputs an expression which could be highly desirable, because it would allow, as an example, OData syntax to be applied to not only REST, but to GRPC by simply adding the additional properties (as strings for the expressions and the real types for take, skip/count) to your request and now you can create an easy mechanism to implement OData with it.