search

payara / Payara

Payara Server is an open source middleware platform that supports reliable and secure deployments of Java EE (Jakarta EE) and MicroProfile applications in any environment: on premise, in the cloud or hybrid.
http://www.payara.fish
Other
886 stars 307 forks source link

Bug Report: Exceptions when activating JMX monitoring in Payara Micro / FISH-6363 #5809

Open Absurd-Mind opened 2 years ago

Absurd-Mind commented 2 years ago

Description

Exceptions when enabling jmx-monitoring in payara micro

Expected Outcome

Enabling JMX-Monitoring should start without exceptions

Current Outcome

Exception 1:

[2022-06-15T10:57:33.041+0200] [] [←[1;93mWARNING←[0m] [] [←[1;94morg.glassfish.flashlight.datatree.impl.MethodInvokerImpl←[0m] [tid: _ThreadID=176 _ThreadName=GF Domain XML] [timeMillis: 1655283453041] [levelValue: 900] [[

java.lang.reflect.InvocationTargetException
        at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:119)
        at java.base/java.lang.reflect.Method.invoke(Method.java:577)
        at org.glassfish.flashlight.datatree.impl.MethodInvokerImpl.getValue(MethodInvokerImpl.java:90)
        at org.glassfish.admin.monitor.StatsProviderManagerDelegateImpl.resetChildNodeStatistics(StatsProviderManagerDelegateImpl.java:594)
        at org.glassfish.admin.monitor.StatsProviderManagerDelegateImpl.resetStatistics(StatsProviderManagerDelegateImpl.java:577)
        at org.glassfish.admin.monitor.StatsProviderManagerDelegateImpl.enableStatsProvider(StatsProviderManagerDelegateImpl.java:419)
        at org.glassfish.admin.monitor.StatsProviderManagerDelegateImpl.enableStatsProviders(StatsProviderManagerDelegateImpl.java:354)
        at org.glassfish.admin.monitor.MonitoringBootstrap.handleLevelChange(MonitoringBootstrap.java:587)
        at org.glassfish.admin.monitor.MonitoringBootstrap.changed(MonitoringBootstrap.java:515)
        at org.jvnet.hk2.config.Transactions$ConfigListenerJob.process(Transactions.java:402)
        at org.jvnet.hk2.config.Transactions$ConfigListenerJob.process(Transactions.java:392)
        at org.jvnet.hk2.config.Transactions$ConfigListenerNotifier$1$1.call(Transactions.java:281)
        at org.jvnet.hk2.config.Transactions$ConfigListenerNotifier$1$1.call(Transactions.java:279)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
        at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: java.lang.StringIndexOutOfBoundsException: Range [0, -1) out of bounds for length 0
        at java.base/jdk.internal.util.Preconditions$1.apply(Preconditions.java:55)
        at java.base/jdk.internal.util.Preconditions$1.apply(Preconditions.java:52)
        at java.base/jdk.internal.util.Preconditions$4.apply(Preconditions.java:213)
        at java.base/jdk.internal.util.Preconditions$4.apply(Preconditions.java:210)
        at java.base/jdk.internal.util.Preconditions.outOfBounds(Preconditions.java:98)
        at java.base/jdk.internal.util.Preconditions.outOfBoundsCheckFromToIndex(Preconditions.java:112)
        at java.base/jdk.internal.util.Preconditions.checkFromToIndex(Preconditions.java:349)
        at java.base/java.lang.AbstractStringBuilder.substring(AbstractStringBuilder.java:1057)
        at java.base/java.lang.StringBuilder.substring(StringBuilder.java:91)
        at org.glassfish.admin.monitor.jvm.JVMRuntimeStatsProvider.getInputArguments(JVMRuntimeStatsProvider.java:137)
        at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104)
        ... 16 more
]]

[2022-06-15T10:57:33.053+0200] [] [←[1;93mWARNING←[0m] [] [←[1;94morg.glassfish.admin.amx.core.AMXValidator←[0m] [tid: _ThreadID=119 _ThreadName=ComplianceMonitor.ValidatorThread] [timeMillis: 1655283453053] [levelValue: 900] [[
  Attribute 'inputarguments-current' failed for amx:pp=/mon/server-mon[server],type=runtime-mon,name=jvm/runtime
java.lang.StringIndexOutOfBoundsException: Range [0, -1) out of bounds for length 0
        at java.base/jdk.internal.util.Preconditions$1.apply(Preconditions.java:55)
        at java.base/jdk.internal.util.Preconditions$1.apply(Preconditions.java:52)
        at java.base/jdk.internal.util.Preconditions$4.apply(Preconditions.java:213)
        at java.base/jdk.internal.util.Preconditions$4.apply(Preconditions.java:210)
        at java.base/jdk.internal.util.Preconditions.outOfBounds(Preconditions.java:98)
        at java.base/jdk.internal.util.Preconditions.outOfBoundsCheckFromToIndex(Preconditions.java:112)
        at java.base/jdk.internal.util.Preconditions.checkFromToIndex(Preconditions.java:349)
        at java.base/java.lang.AbstractStringBuilder.substring(AbstractStringBuilder.java:1057)
        at java.base/java.lang.StringBuilder.substring(StringBuilder.java:91)
        at org.glassfish.admin.monitor.jvm.JVMRuntimeStatsProvider.getInputArguments(JVMRuntimeStatsProvider.java:137)
        at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104)
        at java.base/java.lang.reflect.Method.invoke(Method.java:577)
        at org.glassfish.pfl.basic.facet.FacetAccessorImpl.invoke(FacetAccessorImpl.java:131)
        at org.glassfish.gmbal.impl.MBeanImpl.invoke(MBeanImpl.java:413)
        at org.glassfish.gmbal.impl.AttributeDescriptor.get(AttributeDescriptor.java:110)
        at org.glassfish.gmbal.impl.MBeanSkeleton.getAttribute(MBeanSkeleton.java:526)
        at org.glassfish.gmbal.impl.MBeanImpl.getAttribute(MBeanImpl.java:351)
        at java.management/com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getAttribute(DefaultMBeanServerInterceptor.java:641)
        at java.management/com.sun.jmx.mbeanserver.JmxMBeanServer.getAttribute(JmxMBeanServer.java:679)
        at org.glassfish.admin.amx.util.jmx.MBeanProxyHandler.getAttribute(MBeanProxyHandler.java:238)
        at org.glassfish.admin.amx.core.AMXValidator._validate(AMXValidator.java:490)
        at org.glassfish.admin.amx.core.AMXValidator.validate(AMXValidator.java:1019)
        at org.glassfish.admin.amx.impl.mbean.ComplianceMonitor$ValidatorThread.doRun(ComplianceMonitor.java:226)
        at org.glassfish.admin.amx.impl.mbean.ComplianceMonitor$ValidatorThread.run(ComplianceMonitor.java:205)
]]

Exception 2:

[2022-06-15T10:59:37.212+0200] [] [←[1;93mWARNING←[0m] [NCLS-MON-00504] [←[1;94mjavax.enterprise.system.tools.monitor←[0m] [tid: _ThreadID=192 _ThreadName=GF Domain XML] [timeMillis: 1655283577212] [levelValue: 900] Encountered exception during agent attach

[2022-06-15T10:59:37.213+0200] [] [←[1;93mWARNING←[0m] [NCLS-MON-00511] [←[1;94mjavax.enterprise.system.tools.monitor←[0m] [tid: _ThreadID=192 _ThreadName=GF Domain XML] [timeMillis: 1655283577213] [levelValue: 900] [[
  Error while getting Instrumentation object from ProbeAgentMain
java.lang.ClassNotFoundException: org.glassfish.flashlight.agent.ProbeAgentMain
        at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:641)
        at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:188)
        at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521)
        at org.glassfish.flashlight.impl.client.ProbeProviderClassFileTransformer.<clinit>(ProbeProviderClassFileTransformer.java:450)
        at org.glassfish.flashlight.impl.client.FlashlightProbeClientMediator.transformProbes(FlashlightProbeClientMediator.java:199)
        at org.glassfish.flashlight.impl.client.FlashlightProbeClientMediator.registerListener(FlashlightProbeClientMediator.java:123)
        at org.glassfish.admin.monitor.StatsProviderManagerDelegateImpl.registerStatsProviderToFlashlight(StatsProviderManagerDelegateImpl.java:670)
        at org.glassfish.admin.monitor.StatsProviderManagerDelegateImpl.enableStatsProvider(StatsProviderManagerDelegateImpl.java:424)
        at org.glassfish.admin.monitor.StatsProviderManagerDelegateImpl.enableStatsProviders(StatsProviderManagerDelegateImpl.java:354)
        at org.glassfish.admin.monitor.MonitoringBootstrap.handleLevelChange(MonitoringBootstrap.java:587)
        at org.glassfish.admin.monitor.MonitoringBootstrap.changed(MonitoringBootstrap.java:515)
        at org.jvnet.hk2.config.Transactions$ConfigListenerJob.process(Transactions.java:402)
        at org.jvnet.hk2.config.Transactions$ConfigListenerJob.process(Transactions.java:392)
        at org.jvnet.hk2.config.Transactions$ConfigListenerNotifier$1$1.call(Transactions.java:281)
        at org.jvnet.hk2.config.Transactions$ConfigListenerNotifier$1$1.call(Transactions.java:279)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
        at java.base/java.lang.Thread.run(Thread.java:833)
]]

[2022-06-15T10:59:37.267+0200] [] [←[1;92mINFO←[0m] [NCLS-COM-00027] [←[1;94mjavax.enterprise.system.tools.amx←[0m] [tid: _ThreadID=162 _ThreadName=RMI TCP Connection(4)-172.29.96.1] [timeMillis: 1655283577267] [levelValue: 800] AMX Attribute Change Notification for #1, Wed Jun 15 10:59:37 CEST 2022, "amx:pp=/domain/configs/config[server-config]/monitoring-service,type=module-monitoring-levels", jmx.attribute.change, HttpService: OFF => HIGH

Steps to reproduce

create a file postbootcommands: pbc.env

set-amx-enabled --enabled true --dynamic true --target server
set-jmx-monitoring-configuration --enabled true --dynamic true --amx true --logfrequency 100 --logfrequencyunit MILLISECONDS

set configs.config.server-config.monitoring-service.module-monitoring-levels.jvm=HIGH
set configs.config.server-config.monitoring-service.module-monitoring-levels.http-service=HIGH

Start payara micro: java -jar payara-micro-5.2022.2.jar --postbootcommandfile pbc.env --deploy metrics-example-1.0-SNAPSHOT.war metrics example from: https://github.com/payara/Payara-Examples/tree/master/microprofile/metrics-example

Environment

JamesHillyard commented 2 years ago

Hi @Absurd-Mind,

Payara Server and Payara Micro are not supported on JDK 18, you can read our supported platforms here.

I have however tested your reproducer on JDK 11 and was able to observe the same results, getting the same exceptions as you are facing, so thank you very much for your detailed reproducer. I have raised an internal issue FISH-6363 to address this bug. We always encourage you to contribute, so if you wish you can submit a PR with a fix and we will gladly review it. Otherwise, we will pick up the issue at some point in the future.

Thanks, James