search

eclipse-openj9 / openj9

Eclipse OpenJ9: A Java Virtual Machine for OpenJDK that's optimized for small footprint, fast start-up, and high throughput. Builds on Eclipse OMR (https://github.com/eclipse/omr) and combines with the Extensions for OpenJDK for OpenJ9 repo.
Other
3.28k stars 721 forks source link

IBM HealthCenter agent does not run on OpenJ9 #1538

Open pshipton opened 6 years ago

pshipton commented 6 years ago

Tried it on Windows, installing the wa64-3.0.14.20171017 agent.

java -Xhealthcenter results in the following 

Mar 26, 2018 10:58:35 AM com.ibm.java.diagnostics.healthcenter.agent.mbean.HCLaunchMBean <init>
INFO: Agent version "3.0.14.20171017"
Mar 26, 2018 10:58:35 AM com.ibm.java.diagnostics.healthcenter.agent.mbean.HCLaunchMBean createJMXConnector
INFO: IIOP will be listening on the next available system assigned port. Use com.ibm.java.diagnostics.healthcenter.agent.iiop.port to specify a port
Mar 26, 2018 10:58:35 AM java.io.ObjectInputStream filterCheck
INFO: ObjectInputFilter REJECTED: class javax.rmi.CORBA.Stub, array length: -1, nRefs: 5, depth: 2, bytes: 157, ex: n/a
Mar 26, 2018 10:58:35 AM com.ibm.java.diagnostics.healthcenter.agent.mbean.HCLaunchMBean createJMXConnector
SEVERE: Health Center agent failed to start.
java.io.IOException: Cannot bind to URL [rmi://MYLAPTOP:1972/jmxrmi]: javax.naming.CommunicationException [Root exception is java.rmi.ServerException: RemoteException occurred in server thread; nested exception is:
        java.rmi.UnmarshalException: error unmarshalling arguments; nested exception is:
        java.io.InvalidClassException: filter status: REJECTED]
        at javax.management.remote.rmi.RMIConnectorServer.newIOException(RMIConnectorServer.java:828)
        at javax.management.remote.rmi.RMIConnectorServer.start(RMIConnectorServer.java:433)
        at com.ibm.java.diagnostics.healthcenter.agent.mbean.HCLaunchMBean.createJMXConnector(HCLaunchMBean.java:678)
        at com.ibm.java.diagnostics.healthcenter.agent.mbean.HCLaunchMBean.startMBeanServer(HCLaunchMBean.java:462)
        at com.ibm.java.diagnostics.healthcenter.agent.mbean.HCLaunchMBean$JmxStarterThread.run(HCLaunchMBean.java:267)
Caused by: javax.naming.CommunicationException [Root exception is java.rmi.ServerException: RemoteException occurred in server thread; nested exception is:
        java.rmi.UnmarshalException: error unmarshalling arguments; nested exception is:
        java.io.InvalidClassException: filter status: REJECTED]
        at com.sun.jndi.rmi.registry.RegistryContext.bind(RegistryContext.java:161)
        at com.sun.jndi.toolkit.url.GenericURLContext.bind(GenericURLContext.java:228)
        at javax.naming.InitialContext.bind(InitialContext.java:425)
        at javax.management.remote.rmi.RMIConnectorServer.bind(RMIConnectorServer.java:645)
        at javax.management.remote.rmi.RMIConnectorServer.start(RMIConnectorServer.java:428)
        ... 3 more
Caused by: java.rmi.ServerException: RemoteException occurred in server thread; nested exception is:
        java.rmi.UnmarshalException: error unmarshalling arguments; nested exception is:
        java.io.InvalidClassException: filter status: REJECTED
        at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:393)
        at sun.rmi.transport.Transport$1.run(Transport.java:200)
        at sun.rmi.transport.Transport$1.run(Transport.java:197)
        at java.security.AccessController.doPrivileged(AccessController.java:732)
        at sun.rmi.transport.Transport.serviceCall(Transport.java:196)
        at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:568)
        at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:826)
        at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$0(TCPTransport.java:683)
        at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$$Lambda$7.00000000140AEBA0.run(Unknown Source)
        at java.security.AccessController.doPrivileged(AccessController.java:666)
        at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:682)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:812)
        at sun.rmi.transport.StreamRemoteCall.exceptionReceivedFromServer(StreamRemoteCall.java:283)
        at sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:260)
        at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:375)
        at sun.rmi.registry.RegistryImpl_Stub.bind(RegistryImpl_Stub.java:68)
        at com.sun.jndi.rmi.registry.RegistryContext.bind(RegistryContext.java:155)
        ... 7 more
Caused by: java.rmi.UnmarshalException: error unmarshalling arguments; nested exception is:
        java.io.InvalidClassException: filter status: REJECTED
        at sun.rmi.registry.RegistryImpl_Skel.dispatch(RegistryImpl_Skel.java:78)
        at sun.rmi.server.UnicastServerRef.oldDispatch(UnicastServerRef.java:472)
        at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:299)
        at sun.rmi.transport.Transport$1.run(Transport.java:200)
        at sun.rmi.transport.Transport$1.run(Transport.java:197)
        at java.security.AccessController.doPrivileged(AccessController.java:732)
        at sun.rmi.transport.Transport.serviceCall(Transport.java:196)
        at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:568)
        at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:826)
        at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$0(TCPTransport.java:683)
        at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$$Lambda$7.00000000140AEBA0.run(Unknown Source)
        at java.security.AccessController.doPrivileged(AccessController.java:666)
        at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:682)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:812)
Caused by: java.io.InvalidClassException: filter status: REJECTED
        at java.io.ObjectInputStream.filterCheck(ObjectInputStream.java:1355)
        at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1975)
        at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1851)
        at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1982)
        at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1851)
        at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2139)
        at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1673)
        at java.io.ObjectInputStream.readObjectImpl(ObjectInputStream.java:514)
        at java.io.ObjectInputStream.readObject(ObjectInputStream.java:452)
        at sun.rmi.registry.RegistryImpl_Skel.dispatch(RegistryImpl_Skel.java:76)
        ... 15 more

Mar 26, 2018 10:58:35 AM com.ibm.java.diagnostics.healthcenter.agent.mbean.HCLaunchMBean startAgent
INFO: Health Center agent did not start correctly.
pshipton commented 6 years ago

Using the same version on IBM Java works

Mar 26, 2018 6:28:30 PM com.ibm.java.diagnostics.healthcenter.agent.mbean.HCLaunchMBean <init>
INFO: Agent version "3.0.14.20171017"
Mar 26, 2018 6:28:31 PM com.ibm.java.diagnostics.healthcenter.agent.mbean.HCLaunchMBean createJMXConnector
INFO: IIOP will be listening on the next available system assigned port. Use com.ibm.java.diagnostics.healthcenter.agent.iiop.port to specify a port
Mar 26, 2018 6:28:31 PM com.ibm.java.diagnostics.healthcenter.agent.mbean.HCLaunchMBean startAgent
INFO: Health Center agent started on port 1972.
DanHeidinga commented 6 years ago

The error looks very similar to: https://www.ibm.com/developerworks/community/forums/html/topic?id=ac3e5db4-fba9-4348-a213-2a08feb6625d&ps=10

and https://xap-issues.atlassian.net/browse/XAP-13167 which offers a potential workaround which massaged a bit to the following might work:

-Dsun.rmi.registry.registryFilter=java.**;javax.**;com.ibm.**
pshipton commented 6 years ago

-Dsun.rmi.registry.registryFilter=java.**;javax.**;com.ibm.**

works

 bin/java "-Dsun.rmi.registry.registryFilter=java.**;javax.**;com.ibm.**" -Xhealthcenter
Mar 26, 2018 11:27:18 PM com.ibm.java.diagnostics.healthcenter.agent.mbean.HCLaunchMBean <init>
INFO: Agent version "3.0.14.20171017"
Mar 26, 2018 11:27:18 PM com.ibm.java.diagnostics.healthcenter.agent.mbean.HCLaunchMBean createJMXConnector
INFO: IIOP will be listening on the next available system assigned port. Use com.ibm.java.diagnostics.healthcenter.agent.iiop.port to specify a port
Mar 26, 2018 11:27:18 PM com.ibm.java.diagnostics.healthcenter.agent.mbean.HCLaunchMBean startAgent
INFO: Health Center agent started on port 1972.
pshipton commented 6 years ago

works

I didn't notice earlier, but the OpenJ9 VM didn't exit when the healthcenter agent was started, but it should have.

pshipton commented 6 years ago

-Dsun.rmi.registry.registryFilter=javax.** is sufficient to avoid the stack trace initially reported.

sxa commented 6 years ago

Tagging in @tobespc for awareness

pshipton commented 5 years ago

Tried an OpenJ9 0.11 build and the status has not changed. The workaround -Dsun.rmi.registry.registryFilter=javax.** is still required for the healthcenter agent to start. With this workaround the JVM starts, but does not exit/stop.

pshipton commented 5 years ago

I'll just mention here that somebody noted there isn't currently any healthcenter agent for OSX.

kgibm commented 5 years ago

Note that in my tests at https://github.com/AdoptOpenJDK/openjdk-build/issues/692#issuecomment-476370646, -Dsun.rmi.registry.registryFilter* was not needed and the JVM does stop gracefully.

Flandoll commented 5 years ago

@kgibm I seem to need -Dsun.rmi.registry.registryFilter=javax.** when using jdk8 on win10 system, and the process will not stop automatically. If I use jdk12 and add the system variable CLASSPATH=F:\wa64\lib\ext\healthcenter.jar then run java -Xhealthcenter -version will get an error

PS D:\AdoptOpenJDK\jdk-12+33\bin> ./java -Xhealthcenter -version
3月 27, 2019 12:04:48 上午 com.ibm.java.diagnostics.healthcenter.agent.mbean.HCLaunchMBean <init>
信息: Agent version "3.0.11.20161017"
3月 27, 2019 12:04:48 上午 com.ibm.java.diagnostics.healthcenter.agent.mbean.HCLaunchMBean createJMXConnector
信息: IIOP will be listening on the next available system assigned port. Use com.ibm.java.diagnostics.healthcenter.agent.iiop.port to specify a port
3月 27, 2019 12:04:48 上午 com.ibm.java.diagnostics.healthcenter.agent.mbean.HCLaunchMBean createJMXConnector
严重: Health Center agent failed to start.
java.net.MalformedURLException: Unsupported protocol: iiop
        at java.management/javax.management.remote.JMXConnectorServerFactory.newJMXConnectorServer(JMXConnectorServerFactory.java:320)
        at com.ibm.java.diagnostics.healthcenter.agent.mbean.HCLaunchMBean.createJMXConnector(HCLaunchMBean.java:673)
        at com.ibm.java.diagnostics.healthcenter.agent.mbean.HCLaunchMBean.startMBeanServer(HCLaunchMBean.java:462)
        at com.ibm.java.diagnostics.healthcenter.agent.mbean.HCLaunchMBean$JmxStarterThread.run(HCLaunchMBean.java:267)

3月 27, 2019 12:04:48 上午 com.ibm.java.diagnostics.healthcenter.agent.mbean.HCLaunchMBean startAgent
信息: Health Center agent did not start correctly.
openjdk version "12" 2019-03-19
OpenJDK Runtime Environment AdoptOpenJDK (build 12+33)
Eclipse OpenJ9 VM AdoptOpenJDK (build openj9-0.13.0, JRE 12 Windows 10 amd64-64-Bit Compressed References 20190320_32 (JIT enabled, AOT enabled)
OpenJ9   - caeb51f87
OMR      - 33a33ff2
JCL      - 419f571019 based on jdk-12+33)
kgibm commented 5 years ago

@Flandoll It's not beautiful, but you could try -Xbootclasspath instead of CLASSPATH as per my comment in the other issue.

pshipton commented 4 years ago

FWIW, this is what I did to get HealthCenter running on a Java 11 build. It also works to get the latest agent for OpenJ9 Java 8, just copy the files to the same directories (i.e. jre/bin to jre/bin, jre/lib to jre/lib, jre/lib/ext to jre/lib/ext). On Java 8, changing from iiop isn't required, nor the -Xbootclasspath/a: option.

Download a recent IBM Java 8 build from https://developer.ibm.com/javasdk/downloads/sdk8/

Copy the following files from the Java 8 build to the Java 11 build. jre/bin/healthcenter.dll to bin jre/bin/hcapiplugin.dll to bin jre/lib/healthcenter.properties to lib jre/lib/ext/healthcenter.jar to lib

Edit the healthcenter.properties file and set com.ibm.java.diagnostics.healthcenter.agent.transport=jrmp since iiop doesn't work for Java 11. The configuration reference is https://www.ibm.com/support/knowledgecenter/en/SS3KLZ/com.ibm.java.diagnostics.healthcenter.doc/topics/configproperties.html

Enable it as follows bin\java -Xhealthcenter -Xbootclasspath/a:lib/healthcenter.jar -Dsun.rmi.registry.registryFilter=javax.**

keithc-ca commented 3 years ago

See https://github.com/eclipse/openj9/pull/12262 and related pull requests.

kgibm commented 2 years ago

Copy the following files from the Java 8 build to the Java 11 build. jre/bin/healthcenter.dll to bin jre/bin/hcapiplugin.dll to bin jre/lib/healthcenter.properties to lib jre/lib/ext/healthcenter.jar to lib

Another user hit some issues trying the workaround on Linux:

  1. The first two copies should be to lib rather than bin on Linux, and
  2. -Xbootclasspath/a must be an absolute path if java isn't started from $JAVA_HOME

Example with OpenLiberty and Semeru 11 on Linux:

$ podman run --rm -v /mnt/host:/mnt/host --user root -it icr.io/appcafe/open-liberty:full-java11-openj9-ubi bash
[...]
$ cp -r /mnt/host/work/ibmjava/linux/8.0.7.16/jre/ibm-java-x86_64-80/ /tmp/
$ cp /tmp/ibm-java-x86_64-80/jre/lib/amd64/libhealthcenter.so /opt/java/openjdk/lib/
$ cp /tmp/ibm-java-x86_64-80/jre/lib/amd64/libhcapiplugin.so /opt/java/openjdk/lib/
$ cp /tmp/ibm-java-x86_64-80/jre/lib/healthcenter.properties /opt/java/openjdk/lib/
$ cp /tmp/ibm-java-x86_64-80/jre/lib/ext/healthcenter.jar /opt/java/openjdk/lib/
$ printf -- "-Xhealthcenter:level=headless\n-Xbootclasspath/a:/opt/java/openjdk/lib/healthcenter.jar\n-Dsun.rmi.registry.registryFilter=javax.**\n" > /config/jvm.options
$ sed -i 's/transport=iiop/transport=jrmp/g' /opt/java/openjdk/lib/healthcenter.properties
$ /opt/ol/wlp/bin/server run

[Tue 18 Oct 2022 04:06:22 PM UTC] com.ibm.diagnostics.healthcenter.headless INFO: 4.0.1