Closed fly7632785 closed 2 years ago
To ensure that Micrometer is the culprit, does the JVM stay responsive when disabling the actuator?
I noticed the com.ibm
in the stack trace, do you get the same hangs when running on OpenJDK?
I seem to recall there beeing a performance impact trying to list the threadstates. You might consider disabling the JvmThreadMetrics
(by disabling the JvmMetricsAutoConfiguration
)
Let me know how that behaves.
@checketts 1、It's normal when we remove the Micrometer and let it run for a period of time. So I think it maybe related to the Micrometer. 2、Yes, we use OpenJ9 JVM, but we ruled out this factor. Using normal JAVA 8 Hotpot JVM, it still happen. 3、we will try to disable the ThreadMetrics, see what it perform. thank you for your reply.
PS:Our project may use too many thread pools and theads, don't know if it's related.
Yes, I suspect the issue is the thread state metrics. As @checketts suggested, if you try with not binding JvmThreadMetrics
it should help us narrow down the issue.
Hi, I tried the solution that you provided and disabled JvmThreadMetrics. Running for a period time(from then to now ), it works well. The NOT RESPONSE haven't appeared yet. Maybe it is related to The JvmThreadMetrics. I will continue to observe @shakuzen @checketts
Duplicate of #1805
Describe the bug Hello, we got a problem, our service is not responsed. Need help, thank you.
After using micrometer for a period of time, service is not reponsing. http://xxxxx/actuator/prometheus is not responsing, too.
So, can't continue to collect data
And our any request is not responsing,too. Cause our project system is not useful.
We dump javacore.txt to analysis, but not figure out anything useful. Just find so many threads which are runnable and related to micrometer and promethues
We try use jstat jstack to anylysis, but jstack -l xx is not responsing, too. So we have to kill -3 xx and get javacore.txt to analysis.
This is javacore.txt,hope some people help, thank you in advance! We are not so professional that we can't find the reason
javacore.txt
Environment
PS:Our project may use too many thread pools and theads, don't know if it's related.