npgsql / efcore.pg

Entity Framework Core provider for PostgreSQL
PostgreSQL License
1.59k stars 227 forks source link

Mapping enum issues with multiple DbContext's #3375

Open LucHeart opened 2 weeks ago

LucHeart commented 2 weeks ago

When using multiple DbContexts with the same connection string or data source and trying to .MapEnum the enums present on both of them will throw an exception at startup. We ran into this issue while trying to upgrade from net8 to net9

We are using DbContextPool and PooledDbContextFactory in our application, because in same specific parts it makes more sense to use the factory.

        services.AddDbContextPool<MyContext>(builder =>
        {
            builder.UseNpgsql(config.Db.Conn, optionsBuilder =>
            {
                optionsBuilder.MapEnum<RankType>();
            });
        });

        services.AddPooledDbContextFactory<MyContext>(builder =>
        {
            builder.UseNpgsql(config.Db.Conn, optionsBuilder =>
            {
                optionsBuilder.MapEnum<RankType>();
            });
        });

The same is true when using a datasource. (It really doesnt make sense to me that .MapEnum on the datasource doesnt do anything btw. not sure if this is intended. I assumed just defining it on the data source was enough to begin with)

        var dataSource = new NpgsqlDataSourceBuilder(config.Db.Conn).MapEnum<RankType>().Build();

        services.AddDbContextPool<MyContext>(builder =>
        {
            builder.UseNpgsql(dataSource, optionsBuilder =>
            {
                optionsBuilder.MapEnum<RankType>();
            });
        });

        services.AddPooledDbContextFactory<MyContext>(builder =>
        {
            builder.UseNpgsql(dataSource, optionsBuilder =>
            {
                optionsBuilder.MapEnum<RankType>();
            });
        });

The error that is being thrown

[11:19:13.489] [ERR] [Microsoft.Extensions.Hosting.Internal.Host] Hosting failed to start
System.InvalidOperationException: Sequence contains more than one matching element
   at System.Linq.ThrowHelper.ThrowMoreThanOneMatchException()
   at System.Linq.Enumerable.TryGetSingle[TSource](IEnumerable`1 source, Func`2 predicate, Boolean& found)
   at System.Linq.Enumerable.SingleOrDefault[TSource](IEnumerable`1 source, Func`2 predicate)
   at Npgsql.EntityFrameworkCore.PostgreSQL.Storage.Internal.NpgsqlTypeMappingSource.FindEnumMapping(RelationalTypeMappingInfo& mappingInfo)
   at Npgsql.EntityFrameworkCore.PostgreSQL.Storage.Internal.NpgsqlTypeMappingSource.FindMapping(RelationalTypeMappingInfo& mappingInfo)
   at Microsoft.EntityFrameworkCore.Storage.RelationalTypeMappingSource.<>c.<FindMappingWithConversion>b__8_0(ValueTuple`4 k, RelationalTypeMappingSource self)
   at System.Collections.Concurrent.ConcurrentDictionary`2.GetOrAdd[TArg](TKey key, Func`3 valueFactory, TArg factoryArgument)
   at Microsoft.EntityFrameworkCore.Storage.RelationalTypeMappingSource.FindMappingWithConversion(RelationalTypeMappingInfo mappingInfo, Type providerClrType, ValueConverter customConverter)
   at Microsoft.EntityFrameworkCore.Storage.RelationalTypeMappingSource.FindMapping(MemberInfo member, IModel model, Boolean useAttributes)
   at Microsoft.EntityFrameworkCore.Metadata.Internal.MemberClassifier.IsCandidatePrimitiveProperty(MemberInfo memberInfo, IConventionModel model, Boolean useAttributes, CoreTypeMapping& typeMapping)
   at Microsoft.EntityFrameworkCore.Metadata.Conventions.PropertyDiscoveryConvention.IsCandidatePrimitiveProperty(MemberInfo memberInfo, IConventionTypeBase structuralType, CoreTypeMapping& mapping)
   at Microsoft.EntityFrameworkCore.Metadata.Conventions.PropertyDiscoveryConvention.DiscoverPrimitiveProperties(IConventionTypeBaseBuilder structuralTypeBuilder, IConventionContext context)
   at Microsoft.EntityFrameworkCore.Metadata.Conventions.PropertyDiscoveryConvention.ProcessEntityTypeAdded(IConventionEntityTypeBuilder entityTypeBuilder, IConventionContext`1 context)
   at Microsoft.EntityFrameworkCore.Metadata.Conventions.Internal.ConventionDispatcher.ImmediateConventionScope.OnEntityTypeAdded(IConventionEntityTypeBuilder entityTypeBuilder)
   at Microsoft.EntityFrameworkCore.Metadata.Conventions.Internal.ConventionDispatcher.OnEntityTypeAddedNode.Run(ConventionDispatcher dispatcher)
   at Microsoft.EntityFrameworkCore.Metadata.Conventions.Internal.ConventionDispatcher.DelayedConventionScope.Run(ConventionDispatcher dispatcher)
   at Microsoft.EntityFrameworkCore.Metadata.Conventions.Internal.ConventionDispatcher.ConventionBatch.Run()
   at Microsoft.EntityFrameworkCore.Metadata.Conventions.Internal.ConventionDispatcher.ConventionBatch.Dispose()
   at Microsoft.EntityFrameworkCore.Metadata.Conventions.Internal.ConventionDispatcher.ImmediateConventionScope.OnModelInitialized(IConventionModelBuilder modelBuilder)
   at Microsoft.EntityFrameworkCore.Metadata.Conventions.Internal.ConventionDispatcher.ImmediateConventionScope.OnModelInitialized(IConventionModelBuilder modelBuilder)
   at Microsoft.EntityFrameworkCore.Metadata.Conventions.Internal.ConventionDispatcher.OnModelInitialized(IConventionModelBuilder modelBuilder)
   at Microsoft.EntityFrameworkCore.Metadata.Internal.Model..ctor(ConventionSet conventions, ModelDependencies modelDependencies, ModelConfiguration modelConfiguration)
   at Microsoft.EntityFrameworkCore.ModelBuilder..ctor(ConventionSet conventions, ModelDependencies modelDependencies, ModelConfiguration modelConfiguration)
   at Microsoft.EntityFrameworkCore.ModelConfigurationBuilder.CreateModelBuilder(ModelDependencies modelDependencies)
   at Microsoft.EntityFrameworkCore.Infrastructure.ModelSource.CreateModel(DbContext context, IConventionSetBuilder conventionSetBuilder, ModelDependencies modelDependencies)
   at Microsoft.EntityFrameworkCore.Infrastructure.ModelSource.GetModel(DbContext context, ModelCreationDependencies modelCreationDependencies, Boolean designTime)
   at Microsoft.EntityFrameworkCore.Internal.DbContextServices.CreateModel(Boolean designTime)
   at Microsoft.EntityFrameworkCore.Internal.DbContextServices.get_Model()
   at Microsoft.EntityFrameworkCore.Infrastructure.EntityFrameworkServicesBuilder.<>c.<TryAddCoreServices>b__8_4(IServiceProvider p)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSiteMain(ServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitCache(ServiceCallSite callSite, RuntimeResolverContext context, ServiceProviderEngineScope serviceProviderEngine, RuntimeResolverLock lockType)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitScopeCache(ServiceCallSite callSite, RuntimeResolverContext context)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(ServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitConstructor(ConstructorCallSite constructorCallSite, RuntimeResolverContext context)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSiteMain(ServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitCache(ServiceCallSite callSite, RuntimeResolverContext context, ServiceProviderEngineScope serviceProviderEngine, RuntimeResolverLock lockType)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitScopeCache(ServiceCallSite callSite, RuntimeResolverContext context)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(ServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitConstructor(ConstructorCallSite constructorCallSite, RuntimeResolverContext context)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSiteMain(ServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitCache(ServiceCallSite callSite, RuntimeResolverContext context, ServiceProviderEngineScope serviceProviderEngine, RuntimeResolverLock lockType)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitScopeCache(ServiceCallSite callSite, RuntimeResolverContext context)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(ServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.Resolve(ServiceCallSite callSite, ServiceProviderEngineScope scope)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.DynamicServiceProviderEngine.<>c__DisplayClass2_0.<RealizeService>b__0(ServiceProviderEngineScope scope)
   at Microsoft.Extensions.DependencyInjection.ServiceProvider.GetService(ServiceIdentifier serviceIdentifier, ServiceProviderEngineScope serviceProviderEngineScope)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.ServiceProviderEngineScope.GetService(Type serviceType)
   at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService(IServiceProvider provider, Type serviceType)
   at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService[T](IServiceProvider provider)
   at Microsoft.EntityFrameworkCore.Infrastructure.EntityFrameworkServicesBuilder.<>c.<TryAddCoreServices>b__8_9(IServiceProvider p)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSiteMain(ServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitCache(ServiceCallSite callSite, RuntimeResolverContext context, ServiceProviderEngineScope serviceProviderEngine, RuntimeResolverLock lockType)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitScopeCache(ServiceCallSite callSite, RuntimeResolverContext context)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(ServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitIEnumerable(IEnumerableCallSite enumerableCallSite, RuntimeResolverContext context)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSiteMain(ServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitCache(ServiceCallSite callSite, RuntimeResolverContext context, ServiceProviderEngineScope serviceProviderEngine, RuntimeResolverLock lockType)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitScopeCache(ServiceCallSite callSite, RuntimeResolverContext context)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(ServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.Resolve(ServiceCallSite callSite, ServiceProviderEngineScope scope)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.DynamicServiceProviderEngine.<>c__DisplayClass2_0.<RealizeService>b__0(ServiceProviderEngineScope scope)
   at Microsoft.Extensions.DependencyInjection.ServiceProvider.GetService(ServiceIdentifier serviceIdentifier, ServiceProviderEngineScope serviceProviderEngineScope)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.ServiceProviderEngineScope.GetService(Type serviceType)
   at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetService[T](IServiceProvider provider)
   at Microsoft.EntityFrameworkCore.DbContext.Microsoft.EntityFrameworkCore.Infrastructure.IResettableService.ResetState()
   at Microsoft.EntityFrameworkCore.Internal.DbContextPool`1.Return(IDbContextPoolable context)
   at Microsoft.EntityFrameworkCore.Internal.DbContextLease.Release()
   at Microsoft.EntityFrameworkCore.Internal.ScopedDbContextLease`1.System.IDisposable.Dispose()
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.ServiceProviderEngineScope.Dispose()
   at Startup.Configure(IApplicationBuilder app, IWebHostEnvironment env, ILoggerFactory loggerFactory, ILogger`1 logger) in F:\Dev\Git\\API\API\Startup.cs:line 189
   at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
   at System.Reflection.MethodBaseInvoker.InvokeDirectByRefWithFewArgs(Object obj, Span`1 copyOfArgs, BindingFlags invokeAttr)
   at System.Reflection.MethodBaseInvoker.InvokeWithFewArgs(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
   at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
   at Microsoft.AspNetCore.Hosting.ConfigureBuilder.Invoke(Object instance, IApplicationBuilder builder)
   at Microsoft.AspNetCore.Hosting.GenericWebHostService.StartAsync(CancellationToken cancellationToken)
   at Microsoft.Extensions.Hosting.Internal.Host.<StartAsync>b__14_1(IHostedService service, CancellationToken token)
   at Microsoft.Extensions.Hosting.Internal.Host.ForeachService[T](IEnumerable`1 services, CancellationToken token, Boolean concurrent, Boolean abortOnFirstException, List`1 exceptions, Func`3 operation)
roji commented 1 week ago

Can you please submit a minimal, runnable code sample? Partial snippets are very rarely enough to understand exactly what a user is doing.

LucHeart commented 1 week ago

Here ya go, simple as that.

using Microsoft.EntityFrameworkCore;

var builder = WebApplication.CreateBuilder(args);

const string connectionString = "Host=localhost;Port=5432;Database=root;Username=root;Password=root"; // This doesnt need to be up, just needs to be a valid conn string

builder.Services.AddDbContextPool<MyContext>(optionsBuilder =>
{
    optionsBuilder.UseNpgsql(connectionString, npgsql =>
    {
        npgsql.MapEnum<FunnyEnum>(); // Map Enum
    });
});

builder.Services.AddPooledDbContextFactory<MyContext>(optionsBuilder =>
{
    optionsBuilder.UseNpgsql(connectionString, npgsql =>
    {
        npgsql.MapEnum<FunnyEnum>(); // Map Enum, but this seems to cause the issue
    });
});

var app = builder.Build();

await using var scope = app.Services.CreateAsyncScope();
var myContext = scope.ServiceProvider.GetRequiredService<MyContext>(); // Request context to initialize
var tryToQuery = await myContext.Test.FirstOrDefaultAsync(); // This will throw an exception

app.Run();

public class MyContext : DbContext
{
    public MyContext() { }
    public MyContext(DbContextOptions<MyContext> options) : base(options) { }
    public virtual DbSet<TestSet> Test { get; set; }
}

public class TestSet
{
    public Guid Id { get; set; }
    public FunnyEnum FunnyEnum { get; set; }
}

public enum FunnyEnum
{
    Yes
}

csproj is just web sdk and npgsql efcore

<Project Sdk="Microsoft.NET.Sdk.Web">

    <PropertyGroup>
        <TargetFramework>net9.0</TargetFramework>
        <Nullable>enable</Nullable>
        <ImplicitUsings>enable</ImplicitUsings>
    </PropertyGroup>

    <ItemGroup>
        <PackageReference Include="Npgsql.EntityFrameworkCore.PostgreSQL" Version="9.0.1" />
    </ItemGroup>

</Project>
erwan-joly commented 3 days ago

Facing the same issue, it seems like the MapEnum options write to a options.EnumDefinitions and options is a singleton INpgsqlSingletonOptions so calling UseNpgsql twice ends up adding the enum twice to the list :/

In my config I'm reusing the same exact npgsql options - I get the same one via a method

services.AddDbContext<DbContext>((servicesProvider, options) => DbContextOptionBuilder(options, servicesProvider), ServiceLifetime.Transient);
services.AddPooledDbContextFactory<DbContext>((servicesProvider, options) => DbContextOptionBuilder(options, servicesProvider));

Sadly the old way to do enums don't seems to work anymore and this cause issues with the new way.

Is there anyway to load the config somehow outside of the dbcontext adds ?

something like

service.AddSingleton<INpgsqlSingletonOptions>(new NpgsqlSingletonOptionsBuilder().TheOptions());
services.AddDbContext<DbContext>(ServiceLifetime.Transient);
services.AddPooledDbContextFactory<DbContext>();
roji commented 3 days ago

Thanks for the minimal repro @LucHeart - I can see the problem happening.

So the way EF configuration works, if you specify both AddDbContextPool() and AddPooledDbContextFactory(), the configuration lambdas in both calls incrementally configure the same options; the context instances you get from both injection methods (direct injection, injection of the context factory) have the same options and behave identically.

This means that your code is effectively the same as duplicate invocation of MapEnum, i.e.:

protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
    => optionsBuilder
        .UseNpgsql("Host=localhost;Username=test;Password=test", o =>
        {
            o.MapEnum<FunnyEnum>();
            o.MapEnum<FunnyEnum>();
        })

I'll think a bit and consult with other EF members to see what we think of this... I could of course add logic to the PG provider that detects duplicate and/or conflicting mapping (keep in mind that the 2nd MapEnum could specify a different store type, or name translator - that would be incompatible and have to throw).

But it seems wrong for users to duplicate the context configuration just because the want to use both AddDbContextPool() and AddPooledDbContextFactory(). So at least as a temporary workaround, I'd recommend doing the following:

services.AddDbContextPool<MyContext>(optionsBuilder =>
    optionsBuilder.UseNpgsql(connectionString, npgsql => npgsql.MapEnum<FunnyEnum>()));

services.AddPooledDbContextFactory<MyContext>(_ => {});

In other words, configure only once - in the first lambda - and do nothing in the second. But as I wrote above, I'll consult with the EF team to understand this better.

/cc @ajcvickers

Minimal repro without ASP.NET and some changes ```c# var connectionString = "Host=localhost;Username=test;Password=test"; var services = new ServiceCollection(); services.AddDbContextPool(optionsBuilder => optionsBuilder.UseNpgsql(connectionString, npgsql => npgsql.MapEnum())); // services.AddPooledDbContextFactory(optionsBuilder => // optionsBuilder.UseNpgsql(connectionString, npgsql // => npgsql.MapEnum())); services.AddPooledDbContextFactory(_ => {}); var serviceProvider = services.BuildServiceProvider(); using var scope = serviceProvider.CreateScope(); var contextNotFromFactory = scope.ServiceProvider.GetRequiredService(); var contextFactory = scope.ServiceProvider.GetRequiredService>(); using var contextFromFactory = contextFactory.CreateDbContext(); await contextNotFromFactory.Database.EnsureDeletedAsync(); await contextNotFromFactory.Database.EnsureCreatedAsync(); _ = await contextNotFromFactory.Test.FirstOrDefaultAsync(); _ = await contextFromFactory.Test.FirstOrDefaultAsync(); public class MyContext : DbContext { public MyContext() { } public MyContext(DbContextOptions options) : base(options) { } public virtual DbSet Test { get; set; } } public class TestSet { public Guid Id { get; set; } public FunnyEnum FunnyEnum { get; set; } } public enum FunnyEnum { Yes } ```
roji commented 1 day ago

From discussion with the EF team: we agree that double configuration is a bad thing (i.e. repeating the same MapEnum twice); we may look into allowing both DbContext and IDbContextFactory to be registered in DI via a single call (https://github.com/dotnet/efcore/issues/26528).

Regardless, we think it's a good idea for the PG provider to handle this scenario better, i.e. detect and ignore duplicate MapEnum invocations, and throw (or possibly do last-one-wins) for incompatible invocations.

LucHeart commented 1 day ago

I agree..

In other words, configure only once - in the first lambda - and do nothing in the second

thats what I've been doing in the meantime, just looks kinda sketchy but works fine

roji commented 1 day ago

FWIW I think repeating the same configuration twice is even more sketchy... :/

LucHeart commented 1 day ago

Yeah that too