search

micronaut-projects / micronaut-core

Micronaut Application Framework
http://micronaut.io
Apache License 2.0
6.08k stars 1.07k forks source link

Regression in ConversionService for concurrent application contexts #7948

Closed yawkat closed 1 year ago

yawkat commented 2 years ago

Issue description

7635 changed the ConversionService.DEFAULT to be reset when an application context is closed. This breaks non-standard conversions (e.g. NettyConverters) when there are concurrent application contexts:

    def "test convert bytebuf after context reset"() {
        given:
        ApplicationContext ctx1 = ApplicationContext.run()
        ApplicationContext ctx2 = ApplicationContext.run()
        ByteBuf buf = Unpooled.wrappedBuffer("foo".bytes)

        expect:
        // works as expected
        ctx1.getBean(ConversionService).convert(buf, String).get() == 'foo'
        ctx2.getBean(ConversionService).convert(buf, String).get() == 'foo'

        when:
        ctx2.stop()

        then:
        // this fails, the converter has been removed by ctx2.stop()
        ctx1.getBean(ConversionService).convert(buf, String).get() == 'foo'

        cleanup:
        buf.release()
        ctx1.close()
    }

@dstepanov mentioned that we should probably fix this in 4.0, but we may need to do this earlier. In particular, this issue lead to a test failure in the aws module: https://github.com/micronaut-projects/micronaut-aws/pull/1452 – the application context for one test was closed late, leading to the conversion service for another test being improperly reset.

I'm not sure of the path forward here. We could revert #7635, but I'm not sure if this regression is worse than the bug that was fixed in that PR. We could also move forward with @dstepanov suggestion of a "wrapping" conversion service for each context, which is the proper way to do this, but I'm not sure if uncoupling the context conversion service would lead to other issues that need a major release.

dstepanov commented 2 years ago

Which particular converter from NettyConverters is used? We might want to implement that solution I mentioned in some backward compatible way, with the idea that ConversionService.DEFAULT is not being used there.

yawkat commented 2 years ago

this test case uses compositeByteBufCharSequenceTypeConverter

dstepanov commented 2 years ago

Those don't rely on the bean context and can be extracted to an additional TypeConverterRegistrar registered using the service loader. I would expect those to survive the purge.

yawkat commented 2 years ago

that seems like a good workaround for 3.6, I will make a PR. thanks!

musketyr commented 1 year ago

What is the actual workaround here? I've tried to inject Environment instead of ConversionService but I still got the following error:

io.micronaut.context.exceptions.BeanInstantiationException: Error instantiating bean of type  [io.micronaut.web.router.DefaultRouter] |  
-- | --
  |   |  
  | Message: Bean of type [class io.micronaut.context.DefaultApplicationContext$RuntimeConfiguredEnvironment] is a manually registered singleton that was registered with the context via BeanContext.registerBean(..) and cannot be created directly |  
  | Path Taken: NettyEmbeddedServer.buildDefaultServer(NettyHttpServerConfiguration configuration) --> new DefaultNettyEmbeddedServerFactory(ApplicationContext applicationContext,[RouteExecutor routeExecutor],MediaTypeCodecRegistry mediaTypeCodecRegistry,StaticResourceResolver staticResourceResolver,ThreadFactory nettyThreadFactory,HttpCompressionStrategy httpCompressionStrategy,EventLoopGroupFactory eventLoopGroupFactory,EventLoopGroupRegistry eventLoopGroupRegistry) --> new RouteExecutor([Router router],BeanContext beanContext,RequestArgumentSatisfier requestArgumentSatisfier,HttpServerConfiguration serverConfiguration,ErrorResponseProcessor errorResponseProcessor,ExecutorSelector executorSelector) --> new DefaultRouter([Collection builders]) |  
yawkat commented 1 year ago

@musketyr that does not seem related to this issue?

musketyr commented 1 year ago

@yawkat the stack trace is very similar to what has been stated in original issue https://github.com/micronaut-projects/micronaut-serialization/issues/237


Error instantiating bean of type  [io.micronaut.web.router.DefaultRouter]

Message: Bean of type [class io.micronaut.context.DefaultApplicationContext$RuntimeConfiguredEnvironment] is a manually registered singleton that was registered with the context via BeanContext.registerBean(..) and cannot be created directly
Path Taken: NettyEmbeddedServer.buildDefaultServer(NettyHttpServerConfiguration configuration) --> new DefaultNettyEmbeddedServerFactory(ApplicationContext applicationContext,[RouteExecutor routeExecutor],MediaTypeCodecRegistry mediaTypeCodecRegistry,StaticResourceResolver staticResourceResolver,ThreadFactory nettyThreadFactory,HttpCompressionStrategy httpCompressionStrategy,EventLoopGroupFactory eventLoopGroupFactory,EventLoopGroupRegistry eventLoopGroupRegistry) --> new RouteExecutor([Router router],BeanContext beanContext,RequestArgumentSatisfier requestArgumentSatisfier,HttpServerConfiguration serverConfiguration,ErrorResponseProcessor errorResponseProcessor,ExecutorSelector executorSelector) --> new DefaultRouter([Collection builders])
io.micronaut.context.exceptions.BeanInstantiationException: Error instantiating bean of type  [io.micronaut.web.router.DefaultRouter]

Message: Bean of type [class io.micronaut.context.DefaultApplicationContext$RuntimeConfiguredEnvironment] is a manually registered singleton that was registered with the context via BeanContext.registerBean(..) and cannot be created directly
Path Taken: NettyEmbeddedServer.buildDefaultServer(NettyHttpServerConfiguration configuration) --> new DefaultNettyEmbeddedServerFactory(ApplicationContext applicationContext,[RouteExecutor routeExecutor],MediaTypeCodecRegistry mediaTypeCodecRegistry,StaticResourceResolver staticResourceResolver,ThreadFactory nettyThreadFactory,HttpCompressionStrategy httpCompressionStrategy,EventLoopGroupFactory eventLoopGroupFactory,EventLoopGroupRegistry eventLoopGroupRegistry) --> new RouteExecutor([Router router],BeanContext beanContext,RequestArgumentSatisfier requestArgumentSatisfier,HttpServerConfiguration serverConfiguration,ErrorResponseProcessor errorResponseProcessor,ExecutorSelector executorSelector) --> new DefaultRouter([Collection builders])
    at app//io.micronaut.context.DefaultBeanContext.resolveByBeanFactory(DefaultBeanContext.java:2367)
    at app//io.micronaut.context.DefaultBeanContext.doCreateBean(DefaultBeanContext.java:2305)
    at app//io.micronaut.context.DefaultBeanContext.doCreateBean(DefaultBeanContext.java:2251)
    at app//io.micronaut.context.DefaultBeanContext.createRegistration(DefaultBeanContext.java:3016)
    at app//io.micronaut.context.SingletonScope.getOrCreate(SingletonScope.java:80)
    at app//io.micronaut.context.DefaultBeanContext.findOrCreateSingletonBeanRegistration(DefaultBeanContext.java:2918)
    at app//io.micronaut.context.DefaultBeanContext.resolveBeanRegistration(DefaultBeanContext.java:2879)
    at app//io.micronaut.context.DefaultBeanContext.resolveBeanRegistration(DefaultBeanContext.java:2853)
    at app//io.micronaut.context.DefaultBeanContext.addCandidateToList(DefaultBeanContext.java:3511)
    at app//io.micronaut.context.DefaultBeanContext.resolveBeanRegistrations(DefaultBeanContext.java:3457)
    at app//io.micronaut.context.DefaultBeanContext.getBeanRegistrations(DefaultBeanContext.java:3427)
    at app//io.micronaut.context.DefaultBeanContext.getBeansOfType(DefaultBeanContext.java:1381)
    at app//io.micronaut.context.AbstractBeanResolutionContext.getBeansOfType(AbstractBeanResolutionContext.java:72)
    at app//io.micronaut.context.AbstractInitializableBeanDefinition.resolveBeansOfType(AbstractInitializableBeanDefinition.java:2161)
    at app//io.micronaut.context.AbstractInitializableBeanDefinition.getBeansOfTypeForConstructorArgument(AbstractInitializableBeanDefinition.java:1437)
    at app//io.micronaut.context.DefaultBeanContext.resolveByBeanFactory(DefaultBeanContext.java:2354)
    at app//io.micronaut.context.DefaultBeanContext.doCreateBean(DefaultBeanContext.java:2305)
    at app//io.micronaut.context.DefaultBeanContext.doCreateBean(DefaultBeanContext.java:2251)
    at app//io.micronaut.context.DefaultBeanContext.createRegistration(DefaultBeanContext.java:3016)
    at app//io.micronaut.context.SingletonScope.getOrCreate(SingletonScope.java:80)
    at app//io.micronaut.context.DefaultBeanContext.findOrCreateSingletonBeanRegistration(DefaultBeanContext.java:2918)
    at app//io.micronaut.context.DefaultBeanContext.resolveBeanRegistration(DefaultBeanContext.java:2879)
    at app//io.micronaut.context.DefaultBeanContext.resolveBeanRegistration(DefaultBeanContext.java:2800)
    at app//io.micronaut.context.DefaultBeanContext.getBean(DefaultBeanContext.java:1617)
    at app//io.micronaut.context.AbstractBeanResolutionContext.getBean(AbstractBeanResolutionContext.java:66)
    at app//io.micronaut.context.AbstractInitializableBeanDefinition.resolveBean(AbstractInitializableBeanDefinition.java:2065)
    at app//io.micronaut.context.AbstractInitializableBeanDefinition.getBeanForConstructorArgument(AbstractInitializableBeanDefinition.java:1297)
    at app//io.micronaut.context.DefaultBeanContext.resolveByBeanFactory(DefaultBeanContext.java:2354)
    at app//io.micronaut.context.DefaultBeanContext.doCreateBean(DefaultBeanContext.java:2305)
    at app//io.micronaut.context.DefaultBeanContext.doCreateBean(DefaultBeanContext.java:2251)
    at app//io.micronaut.context.DefaultBeanContext.createRegistration(DefaultBeanContext.java:3016)
    at app//io.micronaut.context.SingletonScope.getOrCreate(SingletonScope.java:80)
    at app//io.micronaut.context.DefaultBeanContext.findOrCreateSingletonBeanRegistration(DefaultBeanContext.java:2918)
    at app//io.micronaut.context.DefaultBeanContext.resolveBeanRegistration(DefaultBeanContext.java:2879)
    at app//io.micronaut.context.DefaultBeanContext.resolveBeanRegistration(DefaultBeanContext.java:2800)
    at app//io.micronaut.context.DefaultBeanContext.getBean(DefaultBeanContext.java:1617)
    at app//io.micronaut.context.AbstractBeanResolutionContext.getBean(AbstractBeanResolutionContext.java:66)
    at app//io.micronaut.context.AbstractInitializableBeanDefinition.resolveBean(AbstractInitializableBeanDefinition.java:2065)
    at app//io.micronaut.context.AbstractInitializableBeanDefinition.getBeanForConstructorArgument(AbstractInitializableBeanDefinition.java:1297)
    at app//io.micronaut.context.DefaultBeanContext.resolveByBeanFactory(DefaultBeanContext.java:2354)
    at app//io.micronaut.context.DefaultBeanContext.doCreateBean(DefaultBeanContext.java:2305)
    at app//io.micronaut.context.DefaultBeanContext.doCreateBean(DefaultBeanContext.java:2251)
    at app//io.micronaut.context.DefaultBeanContext.createRegistration(DefaultBeanContext.java:3016)
    at app//io.micronaut.context.SingletonScope.getOrCreate(SingletonScope.java:80)
    at app//io.micronaut.context.DefaultBeanContext.findOrCreateSingletonBeanRegistration(DefaultBeanContext.java:2918)
    at app//io.micronaut.context.DefaultBeanContext.resolveBeanRegistration(DefaultBeanContext.java:2879)
    at app//io.micronaut.context.DefaultBeanContext.resolveBeanRegistration(DefaultBeanContext.java:2800)
    at app//io.micronaut.context.DefaultBeanContext.getBean(DefaultBeanContext.java:1617)
    at app//io.micronaut.context.DefaultBeanContext.getBean(DefaultBeanContext.java:1599)
    at app//io.micronaut.context.DefaultBeanContext.resolveByBeanFactory(DefaultBeanContext.java:2354)
    at app//io.micronaut.context.DefaultBeanContext.doCreateBean(DefaultBeanContext.java:2305)
    at app//io.micronaut.context.DefaultBeanContext.doCreateBean(DefaultBeanContext.java:2251)
    at app//io.micronaut.context.DefaultBeanContext.createRegistration(DefaultBeanContext.java:3016)
    at app//io.micronaut.context.SingletonScope.getOrCreate(SingletonScope.java:80)
    at app//io.micronaut.context.DefaultBeanContext.findOrCreateSingletonBeanRegistration(DefaultBeanContext.java:2918)
    at app//io.micronaut.context.DefaultBeanContext.resolveBeanRegistration(DefaultBeanContext.java:2879)
    at app//io.micronaut.context.DefaultBeanContext.resolveBeanRegistration(DefaultBeanContext.java:2800)
    at app//io.micronaut.context.DefaultBeanContext.getBean(DefaultBeanContext.java:1617)
    at app//io.micronaut.context.DefaultBeanContext.getBean(DefaultBeanContext.java:867)
    at app//io.micronaut.context.DefaultBeanContext.getBean(DefaultBeanContext.java:859)
    at app//io.micronaut.test.extensions.AbstractMicronautExtension.beforeClass(AbstractMicronautExtension.java:317)
    at app//io.micronaut.test.extensions.spock.MicronautSpockExtension.lambda$visitSpecAnnotation$3(MicronautSpockExtension.java:108)
    at app//org.spockframework.runtime.extension.MethodInvocation.proceed(MethodInvocation.java:101)
    at app//org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:148)
    at app//org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
    at app//org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141)
    at app//org.spockframework.runtime.model.MethodInfo.invoke(MethodInfo.java:148)
    at app//org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139)
    at app//org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
    at app//org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138)
    at app//org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95)
    at java.base@11.0.17/java.util.ArrayList.forEach(ArrayList.java:1541)
    at app//org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:41)
    at app//org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:155)
    at app//org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
    at app//org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141)
    at app//org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
    at app//org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139)
    at app//org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
    at app//org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138)
    at app//org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95)
    at app//org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:35)
    at app//org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:57)
    at app//org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:54)
    at app//org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:147)
    at app//org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:127)
    at app//org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:90)
    at app//org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:55)
    at app//org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:102)
    at app//org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:54)
    at app//org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:114)
    at app//org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:86)
    at app//org.junit.platform.launcher.core.DefaultLauncherSession$DelegatingLauncher.execute(DefaultLauncherSession.java:86)
    at app//org.junit.platform.launcher.core.SessionPerRequestLauncher.execute(SessionPerRequestLauncher.java:53)
    at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor$CollectAllTestClassesExecutor.processAllTestClasses(JUnitPlatformTestClassProcessor.java:99)
    at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor$CollectAllTestClassesExecutor.access$000(JUnitPlatformTestClassProcessor.java:79)
    at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor.stop(JUnitPlatformTestClassProcessor.java:75)
    at org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.stop(SuiteTestClassProcessor.java:61)
    at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
    at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
    at org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:33)
    at org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:94)
    at org.gradle.api.internal.tasks.testing.worker.TestWorker$3.run(TestWorker.java:193)
    at org.gradle.api.internal.tasks.testing.worker.TestWorker.executeAndMaintainThreadName(TestWorker.java:129)
    at org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:100)
    at org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:60)
    at org.gradle.process.internal.worker.child.ActionExecutionWorker.execute(ActionExecutionWorker.java:56)
    at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:133)
    at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:71)
    at app//worker.org.gradle.process.internal.worker.GradleWorkerMain.run(GradleWorkerMain.java:69)
    at app//worker.org.gradle.process.internal.worker.GradleWorkerMain.main(GradleWorkerMain.java:74)
Caused by: java.lang.UnsupportedOperationException: Bean of type [class io.micronaut.context.DefaultApplicationContext$RuntimeConfiguredEnvironment] is a manually registered singleton that was registered with the context via BeanContext.registerBean(..) and cannot be created directly
    at io.micronaut.context.NoInjectionBeanDefinition.getConstructor(NoInjectionBeanDefinition.java:131)
    at io.micronaut.context.DefaultBeanContext.resolveByBeanDefinition(DefaultBeanContext.java:2315)
    at io.micronaut.context.DefaultBeanContext.doCreateBean(DefaultBeanContext.java:2307)
    at io.micronaut.context.DefaultBeanContext.doCreateBean(DefaultBeanContext.java:2251)
    at io.micronaut.context.DefaultBeanContext.createRegistration(DefaultBeanContext.java:3016)
    at io.micronaut.context.SingletonScope.getOrCreate(SingletonScope.java:80)
    at io.micronaut.context.DefaultBeanContext.findOrCreateSingletonBeanRegistration(DefaultBeanContext.java:2918)
    at io.micronaut.context.DefaultBeanContext.resolveBeanRegistration(DefaultBeanContext.java:2879)
    at io.micronaut.context.DefaultBeanContext.resolveBeanRegistration(DefaultBeanContext.java:2800)
    at io.micronaut.context.DefaultBeanContext.getBean(DefaultBeanContext.java:1617)
    at io.micronaut.context.AbstractBeanResolutionContext.getBean(AbstractBeanResolutionContext.java:66)
    at io.micronaut.context.AbstractInitializableBeanDefinition.resolveBean(AbstractInitializableBeanDefinition.java:2065)
    at io.micronaut.context.AbstractInitializableBeanDefinition.getBeanForConstructorArgument(AbstractInitializableBeanDefinition.java:1297)
    at io.micronaut.context.DefaultBeanContext.resolveByBeanFactory(DefaultBeanContext.java:2354)
    ... 110 more

and it's also triggered by when ConversionService is supposed to be injected. Path:

0 = {AbstractBeanResolutionContext$ConstructorArgumentSegment@7803} "new DeleteEndpointRouteBuilder(ApplicationContext beanContext,UriNamingStrategy uriNamingStrategy,[ConversionService conversionService],EndpointDefaultConfiguration endpointDefaultConfiguration)"
1 = {AbstractBeanResolutionContext$ConstructorArgumentSegment@7804} "new DefaultRouter([Collection builders])"
2 = {AbstractBeanResolutionContext$ConstructorArgumentSegment@7805} "new RouteExecutor([Router router],BeanContext beanContext,RequestArgumentSatisfier requestArgumentSatisfier,HttpServerConfiguration serverConfiguration,ErrorResponseProcessor errorResponseProcessor,ExecutorSelector executorSelector)"
3 = {AbstractBeanResolutionContext$ConstructorArgumentSegment@7806} "new DefaultNettyEmbeddedServerFactory(ApplicationContext applicationContext,[RouteExecutor routeExecutor],MediaTypeCodecRegistry mediaTypeCodecRegistry,StaticResourceResolver staticResourceResolver,ThreadFactory nettyThreadFactory,HttpCompressionStrategy httpCompressionStrategy,EventLoopGroupFactory eventLoopGroupFactory,EventLoopGroupRegistry eventLoopGroupRegistry)"
4 = {AbstractBeanResolutionContext$MethodArgumentSegment@7807} "NettyEmbeddedServer.buildDefaultServer(NettyHttpServerConfiguration configuration)"

or am I missing something? I'm trying to create a reproducer outside of our project but no luck so far.

graemerocher commented 1 year ago

usually this is because you are interacting with a context that has been shutdown already

musketyr commented 1 year ago

Ok, I think I have found the source of the issue even it's not clear to me what are the consequences which causes the exception above. I have found a couple of misconfigured Micronaut Worker jobs which are throwing exceptions from theExecutableMethodProcessor#process method. I guess you can ignore my comments for now but I hope they could be useful for anyone who will be solving the same issue in the future.

yawkat commented 1 year ago

this will hopefully be less confusing in 4.x