search

prometheus / jmx_exporter

A process for exposing JMX Beans via HTTP for Prometheus consumption
Apache License 2.0
3.06k stars 1.2k forks source link

Issue in Integrating jmx_prometheus_javaagent with Jboss #1000

Open hemnath83 opened 2 months ago

hemnath83 commented 2 months ago

We are running on Jboss EAP 7.2.0 and Java 1.8. When i tried to integrate jmx_prometheus_javaagent with Jboss facing in loading class file of logger.

Could not load Logmanager "org.jboss.logmanager.LogManager" java.lang.ClassNotFoundException: org.jboss.logmanager.LogManager at java.net.URLClassLoader.findClass(URLClassLoader.java:382)

However when I comment / remove jmx_prometheus_javaagent-0.17.1.jar then jboss is starting without issue and i checked the logs the class of logmanger is getting loaded properly without jmx agent config.

"JAVA_OPTS=%JAVA_OPTS% -Djava.util.logging.manager=org.jboss.logmanager.LogManager" "JAVA_OPTS=%JAVA_OPTS% -javaagent:C:\T24\Area\Temenos\prometheus-2.54.1.windows-amd64\jmx_prometheus_javaagent-0.17.1.jar=8080:C:\T24\Area\Temenos\prometheus-2.54.1.windows-amd64\config.yaml"

dhoard commented 2 months ago

@hemnath83 This issue is caused because a JMX exporter uses Java logging but doesn't use the application's classpath.

You will need to modify the application classpath so that the jar(s) required to use org.jboss.logmanager.LogManager are available to the Java agent classloader.

Reference: https://github.com/prometheus/jmx_exporter/issues/455

hemnath83 commented 1 month ago

Hi @dhoard ,

Thank you for your response. Please note i tried to add to the classpath by explicitly adding -Xbootpath in my JVM args but still it did not work. Im still getting the same error.

Below are my complete JVM arguments in startup for your reference.

set "JAVA_OPTS=-d64 -Xms6G -Xmx8G -XX:MetaspaceSize=96M -XX:MaxMetaspaceSize=1G -Xss1024m"
rem "JAVA_OPTS=%JAVA_OPTS% -verbose:class"
set "JAVA_OPTS=%JAVA_OPTS% -Xbootclasspath/p:$JBOSS_HOME/modules/system/layers/base/org/jboss/logmanager/main/jboss-logmanager-2.1.5.Final-redhat-00001.jar"
set "JAVA_OPTS=%JAVA_OPTS% -XX:+UseG1GC -XX:-UseGCOverheadLimit -XX:+AggressiveOpts -XX:+UseLargePages -XX:G1ReservePercent=20"
rem # set "JAVA_OPTS=-d64 -Xms16G -Xmx16G -XX:MetaspaceSize=96M -XX:MaxMetaspaceSize=1024M"
rem # If required context, we can anable the context for log&como for jboss
set "JAVA_OPTS=%JAVA_OPTS% -Dtafj.home=%TAFJ_HOME% -Dfile.encoding=UTF-8 -Dtemenos.log.context=browser -Djbc.debug=9797"
set "JAVA_OPTS=%JAVA_OPTS% -Djava.util.logging.manager=org.jboss.logmanager.LogManager"
set "JAVA_OPTS=%JAVA_OPTS% -verbose:class"
set "JAVA_OPTS=%JAVA_OPTS% -javaagent:C:\\T24\\Area\\Temenos\\prometheus-2.54.1.windows-amd64\\jmx_prometheus_javaagent-0.17.1.jar=8080:C:\\T24\\Area\\Temenos\\prometheus-2.54.1.windows-amd64\\config.yaml"
set "JAVA_OPTS=%JAVA_OPTS% -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=E:\\Dumps\\Dump_KJ"
set "JAVA_OPTS=%JAVA_OPTS% -verbose:gc -Xloggc:jboss.server.log.dir.gc_%p_%t.log -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+PrintGCDateStamps -XX:NumberOfGCLogFiles=10 -XX:+UseGCLogFileRotation -XX:GCLogFileSize=50M"

Update: formatting @dhoard

dhoard commented 4 weeks ago

@hemnath83 per https://github.com/prometheus/jmx_exporter/issues/455 is looks like there are multiple jar files required for JBoss logging. You will need to find all jars required.

Can you either define an environment variable or Java system property to capture developer debug? This may provide more information.

environment variable

JMX_PROMETHEUS_EXPORTER_DEVELOPER_DEBUG=true

Java system property

jmx.prometheus.exporter.developer.debug=true