dotnet / efcore

EF Core is a modern object-database mapper for .NET. It supports LINQ queries, change tracking, updates, and schema migrations.
https://docs.microsoft.com/ef/
MIT License
13.78k stars 3.19k forks source link

JsonElement not supported with Sqlite #34752

Open cyungmann opened 1 month ago

cyungmann commented 1 month ago

Getting exception System.InvalidOperationException: 'No coercion operator is defined between types 'System.IO.MemoryStream' and 'System.Text.Json.JsonElement'.' when trying to pull back records with JsonElement properties.

Stacktrace:

   at System.Linq.Expressions.Expression.GetUserDefinedCoercionOrThrow(ExpressionType coercionType, Expression expression, Type convertToType) in /_/src/libraries/System.Linq.Expressions/src/System/Linq/Expressions/UnaryExpression.cs:line 410
   at Microsoft.EntityFrameworkCore.Query.RelationalShapedQueryCompilingExpressionVisitor.ShaperProcessingExpressionVisitor.CreateGetValueExpression(ParameterExpression dbDataReader, Int32 index, Boolean nullable, RelationalTypeMapping typeMapping, Type type, IPropertyBase property) in /_/src/EFCore.Relational/Query/RelationalShapedQueryCompilingExpressionVisitor.ShaperProcessingExpressionVisitor.cs:line 2537
   at Microsoft.EntityFrameworkCore.Query.RelationalShapedQueryCompilingExpressionVisitor.ShaperProcessingExpressionVisitor.VisitMethodCall(MethodCallExpression methodCallExpression) in /_/src/EFCore.Relational/Query/RelationalShapedQueryCompilingExpressionVisitor.ShaperProcessingExpressionVisitor.cs:line 1254
   at System.Linq.Expressions.ExpressionVisitor.VisitBinary(BinaryExpression node) in /_/src/libraries/System.Linq.Expressions/src/System/Linq/Expressions/ExpressionVisitor.cs:line 195
   at Microsoft.EntityFrameworkCore.Query.RelationalShapedQueryCompilingExpressionVisitor.ShaperProcessingExpressionVisitor.VisitBinary(BinaryExpression binaryExpression) in /_/src/EFCore.Relational/Query/RelationalShapedQueryCompilingExpressionVisitor.ShaperProcessingExpressionVisitor.cs:line 500
   at System.Dynamic.Utils.ExpressionVisitorUtils.VisitBlockExpressions(ExpressionVisitor visitor, BlockExpression block) in /_/src/libraries/System.Linq.Expressions/src/System/Dynamic/Utils/ExpressionVisitorUtils.cs:line 15
   at System.Linq.Expressions.ExpressionVisitor.VisitBlock(BlockExpression node) in /_/src/libraries/System.Linq.Expressions/src/System/Linq/Expressions/ExpressionVisitor.cs:line 213
   at System.Dynamic.Utils.ExpressionVisitorUtils.VisitBlockExpressions(ExpressionVisitor visitor, BlockExpression block) in /_/src/libraries/System.Linq.Expressions/src/System/Dynamic/Utils/ExpressionVisitorUtils.cs:line 15
   at System.Linq.Expressions.ExpressionVisitor.VisitBlock(BlockExpression node) in /_/src/libraries/System.Linq.Expressions/src/System/Linq/Expressions/ExpressionVisitor.cs:line 213
   at System.Linq.Expressions.ExpressionVisitor.VisitSwitchCase(SwitchCase node) in /_/src/libraries/System.Linq.Expressions/src/System/Linq/Expressions/ExpressionVisitor.cs:line 473
   at System.Linq.Expressions.ExpressionVisitor.Visit[T](ReadOnlyCollection`1 nodes, Func`2 elementVisitor) in /_/src/libraries/System.Linq.Expressions/src/System/Linq/Expressions/ExpressionVisitor.cs:line 98
   at System.Linq.Expressions.ExpressionVisitor.VisitSwitch(SwitchExpression node) in /_/src/libraries/System.Linq.Expressions/src/System/Linq/Expressions/ExpressionVisitor.cs:line 484
   at System.Linq.Expressions.ExpressionVisitor.VisitBinary(BinaryExpression node) in /_/src/libraries/System.Linq.Expressions/src/System/Linq/Expressions/ExpressionVisitor.cs:line 195
   at Microsoft.EntityFrameworkCore.Query.RelationalShapedQueryCompilingExpressionVisitor.ShaperProcessingExpressionVisitor.VisitBinary(BinaryExpression binaryExpression) in /_/src/EFCore.Relational/Query/RelationalShapedQueryCompilingExpressionVisitor.ShaperProcessingExpressionVisitor.cs:line 500
   at System.Dynamic.Utils.ExpressionVisitorUtils.VisitBlockExpressions(ExpressionVisitor visitor, BlockExpression block) in /_/src/libraries/System.Linq.Expressions/src/System/Dynamic/Utils/ExpressionVisitorUtils.cs:line 15
   at System.Linq.Expressions.ExpressionVisitor.VisitBlock(BlockExpression node) in /_/src/libraries/System.Linq.Expressions/src/System/Linq/Expressions/ExpressionVisitor.cs:line 213
   at System.Linq.Expressions.ExpressionVisitor.VisitConditional(ConditionalExpression node) in /_/src/libraries/System.Linq.Expressions/src/System/Linq/Expressions/ExpressionVisitor.cs:line 232
   at System.Linq.Expressions.ExpressionVisitor.VisitConditional(ConditionalExpression node) in /_/src/libraries/System.Linq.Expressions/src/System/Linq/Expressions/ExpressionVisitor.cs:line 232
   at System.Dynamic.Utils.ExpressionVisitorUtils.VisitBlockExpressions(ExpressionVisitor visitor, BlockExpression block) in /_/src/libraries/System.Linq.Expressions/src/System/Dynamic/Utils/ExpressionVisitorUtils.cs:line 15
   at System.Linq.Expressions.ExpressionVisitor.VisitBlock(BlockExpression node) in /_/src/libraries/System.Linq.Expressions/src/System/Linq/Expressions/ExpressionVisitor.cs:line 213
   at Microsoft.EntityFrameworkCore.Query.RelationalShapedQueryCompilingExpressionVisitor.ShaperProcessingExpressionVisitor.VisitExtension(Expression extensionExpression) in /_/src/EFCore.Relational/Query/RelationalShapedQueryCompilingExpressionVisitor.ShaperProcessingExpressionVisitor.cs:line 620
   at Microsoft.EntityFrameworkCore.Query.RelationalShapedQueryCompilingExpressionVisitor.ShaperProcessingExpressionVisitor.ProcessShaper(Expression shaperExpression, RelationalCommandCache& relationalCommandCache, IReadOnlyList`1& readerColumns, LambdaExpression& relatedDataLoaders, Int32& collectionId) in /_/src/EFCore.Relational/Query/RelationalShapedQueryCompilingExpressionVisitor.ShaperProcessingExpressionVisitor.cs:line 319
   at Microsoft.EntityFrameworkCore.Query.RelationalShapedQueryCompilingExpressionVisitor.VisitShapedQuery(ShapedQueryExpression shapedQueryExpression) in /_/src/EFCore.Relational/Query/RelationalShapedQueryCompilingExpressionVisitor.cs:line 321
   at Microsoft.EntityFrameworkCore.Query.ShapedQueryCompilingExpressionVisitor.VisitExtension(Expression extensionExpression) in /_/src/EFCore/Query/ShapedQueryCompilingExpressionVisitor.cs:line 89
   at Microsoft.EntityFrameworkCore.Query.RelationalShapedQueryCompilingExpressionVisitor.VisitExtension(Expression extensionExpression) in /_/src/EFCore.Relational/Query/RelationalShapedQueryCompilingExpressionVisitor.cs:line 48
   at Microsoft.EntityFrameworkCore.Query.QueryCompilationContext.CreateQueryExecutor[TResult](Expression query) in /_/src/EFCore/Query/QueryCompilationContext.cs:line 169
   at Microsoft.EntityFrameworkCore.Storage.Database.CompileQuery[TResult](Expression query, Boolean async) in /_/src/EFCore/Storage/Database.cs:line 66
   at Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler.CompileQueryCore[TResult](IDatabase database, Expression query, IModel model, Boolean async) in /_/src/EFCore/Query/Internal/QueryCompiler.cs:line 82
   at Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler.<>c__DisplayClass12_0`1.<ExecuteAsync>b__0() in /_/src/EFCore/Query/Internal/QueryCompiler.cs:line 115
   at Microsoft.EntityFrameworkCore.Query.Internal.CompiledQueryCache.GetOrAddQuery[TResult](Object cacheKey, Func`1 compiler) in /_/src/EFCore/Query/Internal/CompiledQueryCache.cs:line 66
   at Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler.ExecuteAsync[TResult](Expression query, CancellationToken cancellationToken) in /_/src/EFCore/Query/Internal/QueryCompiler.cs:line 111
   at Microsoft.EntityFrameworkCore.Query.Internal.EntityQueryProvider.ExecuteAsync[TResult](Expression expression, CancellationToken cancellationToken) in /_/src/EFCore/Query/Internal/EntityQueryProvider.cs:line 83
   at Microsoft.EntityFrameworkCore.EntityFrameworkQueryableExtensions.ExecuteAsync[TSource,TResult](MethodInfo operatorMethodInfo, IQueryable`1 source, Expression expression, CancellationToken cancellationToken) in /_/src/EFCore/Extensions/EntityFrameworkQueryableExtensions.cs:line 3172
   at Microsoft.EntityFrameworkCore.EntityFrameworkQueryableExtensions.ExecuteAsync[TSource,TResult](MethodInfo operatorMethodInfo, IQueryable`1 source, CancellationToken cancellationToken) in /_/src/EFCore/Extensions/EntityFrameworkQueryableExtensions.cs:line 3187
   at Microsoft.EntityFrameworkCore.EntityFrameworkQueryableExtensions.SingleAsync[TSource](IQueryable`1 source, CancellationToken cancellationToken) in /_/src/EFCore/Extensions/EntityFrameworkQueryableExtensions.cs:line 689
   at Program.<<Main>$>d__0.MoveNext() in C:\Users\cyungmann\source\repos\SqliteJsonTest\SqliteJsonTest\Program.cs:line 13
   at Program.<<Main>$>d__0.MoveNext() in C:\Users\cyungmann\source\repos\SqliteJsonTest\SqliteJsonTest\Program.cs:line 16
   at Program.<<Main>$>d__0.MoveNext() in C:\Users\cyungmann\source\repos\SqliteJsonTest\SqliteJsonTest\Program.cs:line 16
   at Program.<Main>(String[] args)

Code is here: SqliteJsonTest.zip

EF Core version: 8.0.8 Database provider: Microsoft.EntityFrameworkCore.Sqlite Target framework: .NET 8.0 Operating system: Windows 10 Enterprise 22H2 IDE: Visual Studio 2022 17.11.4

cyungmann commented 1 month ago

In case it's helpful, here are copies of the code files:

Program.cs

using Microsoft.Data.Sqlite;
using Microsoft.EntityFrameworkCore;
using SqliteJsonTest;

await using var conn = new SqliteConnection(new SqliteConnectionStringBuilder
{
    DataSource = ":memory:",
}.ConnectionString);
await conn.OpenAsync();
await using var db = new WidgetContext(new DbContextOptionsBuilder<WidgetContext>().UseSqlite(conn).Options);
await db.Database.EnsureCreatedAsync();

var widget = await db.Widgets.SingleAsync();

WidgetContext.cs

using Microsoft.EntityFrameworkCore;

namespace SqliteJsonTest;

internal sealed class WidgetContext : DbContext
{
    public DbSet<Widget> Widgets { get; set; }

    public WidgetContext(DbContextOptions options) : base(options)
    {
    }
}

Widget.cs

using System.Text.Json;

namespace SqliteJsonTest;

internal sealed class Widget
{
    public long Id { get; set; }

    public JsonElement Metadata { get; set; }
}

SqliteJsonTest.csproj

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>net8.0</TargetFramework>
    <ImplicitUsings>enable</ImplicitUsings>
    <Nullable>enable</Nullable>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="Microsoft.EntityFrameworkCore.Sqlite" Version="8.0.8" />
  </ItemGroup>

</Project>
maumar commented 1 month ago

related to / dupe of https://github.com/dotnet/efcore/issues/28871 we could consider throwing a better exception in the meantime

cyungmann commented 1 month ago

@maumar using Microsoft.EntityFrameworkCore.Sqlite 7.0.20, I get System.InvalidOperationException: 'The property 'Widget.Metadata' could not be mapped because it is of type 'JsonElement', which is not a supported primitive type or a valid entity type. Either explicitly map this property, or ignore it using the '[NotMapped]' attribute or by using 'EntityTypeBuilder.Ignore' in 'OnModelCreating'.', which seems better.

maumar commented 1 month ago

yeah, we should get back to something similar. Looks like we bypass any validation and try to construct the shaper for JsonElement as if it were a regular POCO entity mapped to json (and resulting in nasty error about MemoryStream).

We do use clr type JsonElement to mark column as json (as opposed to regular string in sqlite/sql server). Maybe that's why validation gets fooled.