search

eclipse-ee4j / openmq

OpenMQ
https://projects.eclipse.org/projects/ee4j.openmq/
Other
52 stars 34 forks source link

Automatic recovery with stomp bridge enables fails with: java.lang.NoClassDefFoundError: Could not initialize class com.sun.messaging.jmq.jmsclient.UnifiedConnectionImpl #341

Open glassfishrobot opened 11 years ago

glassfishrobot commented 11 years ago

When we are killing our glassfish with kill -9 because of the following bug: https://java.net/jira/browse/MQ-340

We are not able anymore to directly restart the server. The following message appears in the server.log:

[#|2013-08-08T08:42:36.216+0000|SEVERE|glassfish3.1.2|javax.enterprise.system.std.com.sun.enterprise.server.logging|_ThreadID=140;_ThreadName=Thread-2;|Exception in thread "Recovery Helper Thread" |#]

[#|2013-08-08T08:42:36.218+0000|SEVERE|glassfish3.1.2|javax.enterprise.system.std.com.sun.enterprise.server.logging|_ThreadID=140;_ThreadName=Thread-2;|java.lang.NoClassDefFoundError: com.sun.messaging.jmq.jmsclient.UnifiedConnectionImpl
    at com.sun.messaging.XAConnectionFactory.createXAConnection(XAConnectionFactory.java:97)
    at com.sun.messaging.jms.ra.ManagedConnection.<init>(ManagedConnection.java:196)
    at com.sun.messaging.jms.ra.ManagedConnectionFactory.createManagedConnection(ManagedConnectionFactory.java:226)
    at com.sun.enterprise.resource.recovery.ConnectorsRecoveryResourceHandler.loadXAResourcesAndItsConnections(ConnectorsRecoveryResourceHandler.java:292)
    at com.sun.enterprise.transaction.jts.ResourceRecoveryManagerImpl.getAllRecoverableResources(ResourceRecoveryManagerImpl.java:210)
    at com.sun.enterprise.transaction.jts.ResourceRecoveryManagerImpl.recoverXAResources(ResourceRecoveryManagerImpl.java:234)
    at com.sun.enterprise.transaction.jts.ResourceRecoveryManagerImpl.recoverXAResources(ResourceRecoveryManagerImpl.java:331)
    at com.sun.enterprise.transaction.jts.ResourceRecoveryManagerImpl.postConstruct(ResourceRecoveryManagerImpl.java:106)
    at com.sun.hk2.component.AbstractCreatorImpl.inject(AbstractCreatorImpl.java:131)
    at com.sun.hk2.component.ConstructorCreator.initialize(ConstructorCreator.java:91)
    at com.sun.hk2.component.AbstractCreatorImpl.get(AbstractCreatorImpl.java:82)
    at com.sun.hk2.component.SingletonInhabitant.get(SingletonInhabitant.java:67)
    at com.sun.hk2.component.EventPublishingInhabitant.get(EventPublishingInhabitant.java:139)
    at com.sun.hk2.component.AbstractInhabitantImpl.get(AbstractInhabitantImpl.java:78)
    at org.jvnet.hk2.component.Habitat.getByContract(Habitat.java:1050)
    at com.sun.jts.jta.TransactionServiceProperties$RecoveryHelperThread.run(TransactionServiceProperties.java:358)
|#]

We are running JMS in embedded mode and it is doing recovery on restart.

The problem here is that the JMS system is started for automatic recovery and the stomp bridge is started too. Unfortunately the classloader for the recovery thread seams to have problems to load UnifiedConnectionImpl.

I also added some details in the following thread: https://www.java.net/forum/topic/glassfish/glassfish/noclassdeffounderror-when-using-stomp-bridge-openmq-one-cluster-node

The following work around works:

  1. Undeployed application
  2. Restarted both nodes
  3. Redeployed application

    Environment

    Glassfish 3.1.2.2, Linux, Java 7

    Affected Versions

    [4.5.2]

glassfishrobot commented 6 years ago
glassfishrobot commented 11 years ago

@glassfishrobot Commented Reported by manuel_b

glassfishrobot commented 11 years ago

@glassfishrobot Commented @amykang2020 said: Could you please provide a reproducible test case (example application, steps) ? Thanks

glassfishrobot commented 11 years ago

@glassfishrobot Commented manuel_b said:

/opt/glassfish3/bin/asadmin stop-local-instance
imq.bridge.enabled=true
imq.bridge.activelist=stomp
imq.instanceconfig.version=300
imq.bridge.admin.password=admin
imq.jmsra.managed=true
imq.bridge.admin.user=admin
/opt/glassfish3/bin/asadmin start-local-instance
Kill local instance with kill -9

Restart local instance

/opt/glassfish3/bin/asadmin start-local-instance

Observe error:

[#|2013-08-08T08:42:36.216+0000|SEVERE|glassfish3.1.2|javax.enterprise.system.std.com.sun.enterprise.server.logging|_ThreadID=140;_ThreadName=Thread-2;|Exception in thread "Recovery Helper Thread" |#]

[#|2013-08-08T08:42:36.218+0000|SEVERE|glassfish3.1.2|javax.enterprise.system.std.com.sun.enterprise.server.logging|_ThreadID=140;_ThreadName=Thread-2;|java.lang.NoClassDefFoundError: com.sun.messaging.jmq.jmsclient.UnifiedConnectionImpl
    at com.sun.messaging.XAConnectionFactory.createXAConnection(XAConnectionFactory.java:97)
    at com.sun.messaging.jms.ra.ManagedConnection.<init>(ManagedConnection.java:196)
    at com.sun.messaging.jms.ra.ManagedConnectionFactory.createManagedConnection(ManagedConnectionFactory.java:226)
    at com.sun.enterprise.resource.recovery.ConnectorsRecoveryResourceHandler.loadXAResourcesAndItsConnections(ConnectorsRecoveryResourceHandler.java:292)
    at com.sun.enterprise.transaction.jts.ResourceRecoveryManagerImpl.getAllRecoverableResources(ResourceRecoveryManagerImpl.java:210)
    at com.sun.enterprise.transaction.jts.ResourceRecoveryManagerImpl.recoverXAResources(ResourceRecoveryManagerImpl.java:234)
    at com.sun.enterprise.transaction.jts.ResourceRecoveryManagerImpl.recoverXAResources(ResourceRecoveryManagerImpl.java:331)
    at com.sun.enterprise.transaction.jts.ResourceRecoveryManagerImpl.postConstruct(ResourceRecoveryManagerImpl.java:106)
    at com.sun.hk2.component.AbstractCreatorImpl.inject(AbstractCreatorImpl.java:131)
    at com.sun.hk2.component.ConstructorCreator.initialize(ConstructorCreator.java:91)
    at com.sun.hk2.component.AbstractCreatorImpl.get(AbstractCreatorImpl.java:82)
    at com.sun.hk2.component.SingletonInhabitant.get(SingletonInhabitant.java:67)
    at com.sun.hk2.component.EventPublishingInhabitant.get(EventPublishingInhabitant.java:139)
    at com.sun.hk2.component.AbstractInhabitantImpl.get(AbstractInhabitantImpl.java:78)
    at org.jvnet.hk2.component.Habitat.getByContract(Habitat.java:1050)
    at com.sun.jts.jta.TransactionServiceProperties$RecoveryHelperThread.run(TransactionServiceProperties.java:358)
|#]
glassfishrobot commented 11 years ago

@glassfishrobot Commented manuel_b said: @amyk unfortunately I was not able to get the local cluster working therefore this is only a theoretical explanation.

glassfishrobot commented 11 years ago

@glassfishrobot Commented manuel_b said: This ticket might be connected to:

https://java.net/jira/browse/GLASSFISH-17971

glassfishrobot commented 11 years ago

@glassfishrobot Commented manuel_b said: I debugged a little further and figured out that the OSGI Bundle Classloader is used for trying for finding this class. This does not work.

glassfishrobot commented 11 years ago

@glassfishrobot Commented manuel_b said: I now copied the following files into /opt/glassfish3/glassfish/lib and it works:

cd /opt/glassfish3/glassfish/lib
cp /opt/glassfish3/glassfish/lib/install/applications/jmsra/*.jar .
glassfishrobot commented 11 years ago

@glassfishrobot Commented manuel_b said: To be concrete I copied the following files:

-rw-rw-r--  1 ec2-user ec2-user   99926 21\. Aug 15:59 fscontext.jar
-rw-rw-r--  1 ec2-user ec2-user 2110762 21\. Aug 15:59 imqbroker.jar
-rw-rw-r--  1 ec2-user ec2-user  521942 21\. Aug 15:42 imq.jar
-rw-rw-r--  1 ec2-user ec2-user  195921 21\. Aug 15:59 imqjmsbridge.jar
-rw-rw-r--  1 ec2-user ec2-user 1525978 21\. Aug 15:59 imqjmsra.jar
-rw-rw-r--  1 ec2-user ec2-user   53723 21\. Aug 15:59 imqjmx.jar
-rw-rw-r--  1 ec2-user ec2-user   98191 21\. Aug 15:59 imqstomp.jar
-rw-rw-r--  1 ec2-user ec2-user  357149 21\. Aug 15:48 imqutil.jar
glassfishrobot commented 11 years ago

@glassfishrobot Commented @nigeldeakin said: Thanks for the report, the clear detail, your investigations and analysis, and the workaround. As you suggest the problem appears to be that the JMSRA resource adapter does not appear to be available to the class loader that is used during this recovery operation. Obviously this should be investigated further.

Your report reminded me of this question on the forum https://www.java.net/forum/topic/glassfish/glassfish/noclassdeffounderror-when-using-stomp-bridge-openmq-one-cluster-node which I now see was from you.

glassfishrobot commented 11 years ago

@glassfishrobot Commented @nigeldeakin said: I agree that https://java.net/jira/browse/GLASSFISH-17971 looks similar. That issue was apparently fixed in GlassFish 3.1.2_b20 Which version of GlassFish were you using?

(Updated) I see the version used was 3.1.2.2 (July 2012). That should contain the fix to GLASSFISH-17971. Will follow up.

glassfishrobot commented 7 years ago

@glassfishrobot Commented This issue was imported from java.net JIRA MQ-341