Open vijayender opened 4 years ago
I ran into this same issue on the newest version of Micronaut (2.3.4).
We have a Singleton bean, that in it's constructor does a HTTP call via a @Client
class.
When the HTTP call returns, some code in micronaut wants to parse the JSON response using Jackson, and calls into some code that tries to get the ObjectMapper singleton bean. This hangs until something times out.
What i think goes on:
new
-ing up our singleton bean. The thread parks while waiting for a response to our HTTP call done in the constructor.new
-ed up by Micronaut as a singleton bean.Important points for repro: The http call done must be one that returns JSON. ObjectMapper must also not have been used anywhere else in the app before the HTTP call returns.
We managed to work around this by defining ObjectMapper as a @Context
scoped bean in our project. If ObjectMapper already exists as a bean when the HTTP request returns, no deadlock happens.
we need to improve error handling for nested bean initialization (i.e. throw an error)
Ok I don't think we can detect such deadlocks reliably. Test case is this:
package io.micronaut.context
import jakarta.annotation.PostConstruct
import jakarta.inject.Inject
import jakarta.inject.Singleton
import spock.lang.Issue
import spock.lang.Specification
import spock.lang.Timeout
class InitializationDeadlockSpec extends Specification {
@Timeout(10)
@Issue('https://github.com/micronaut-projects/micronaut-core/issues/3910')
def 'nested initialization on different threads should not lead to deadlock'() {
given:
def ctx = ApplicationContext.run()
expect:
ctx.getBean(BeanA) != null
ctx.getBean(BeanB).visitedFromA
}
@Singleton
static class BeanA {
@Inject
BeanContext context
@PostConstruct
void createB() {
def thread = new Thread(() -> {
context.getBean(BeanB).visitedFromA = true
})
thread.start()
thread.join()
}
}
@Singleton
static class BeanB {
boolean visitedFromA = false
}
}
@graemerocher is it be feasible to reduce scope of locking in DefaultBeanContext so that this can work? I think that in principle it should be possible to lock only for the specific BeanKey that is being requested (instead of locking on the full singletonObjects), so that other threads can handle other BeanKeys concurrently. But not sure if there are other complications or how difficult this would be to build.
That could be fixed by doing synchronization on the instance of BeanDefinition
in findOrCreateSingletonBeanRegistration
but the problem is that we can have multiple instances of the same BeanDefinition
. The other approach would be to have a map of locks...
@dstepanov when I looked at the CDI implementations out there that do this it appears a map of locks is what they do.
@dstepanov assigned to you, perhaps worth addressing as part of the broader refactor you are doing
Thanks for reporting an issue, please review the task list below before submitting the issue. Your issue report will be closed if the issue is incomplete and the below tasks not completed.
NOTE: If you are unsure about something and the issue is more of a question a better place to ask questions is on Stack Overflow (https://stackoverflow.com/tags/micronaut) or Gitter (https://gitter.im/micronautfw/). DO NOT use the issue tracker to ask questions.
Task List
Steps to Reproduce
Client
that returns a json responseinit{}
orpostConstruct
section make the call viaClient
defined aboveExpected Behaviour
The Bean should get inited
Actual Behaviour
The call is hung My analysis:
Environment Information
Example Application
There is an example app to reproduce the issue:
https://github.com/vijayender/rx-instrumentation-micronaut-issue
Start the python server in it using
python3 test.py
(runs on 8081)Start the server
First call to
http://localhost:8080/tryout
fails with your browsers timeout (in the absence of client timeout can hang forever)To confirm the deadlock analysis: Try uncommenting https://github.com/vijayender/rx-instrumentation-micronaut-issue/blob/master/src/main/kotlin/example/micronaut/Controller.kt#L33 and run again, the first call to
/tryout/
will no more be blocked.