Inbound message processing halts because MaxWaitTime functionality is broken

[arinban]

We notice that occasionally inbound message processing halts on our application servers until we restart the server or redeploy the resource adapter. There is no error message or exception in the logs.

I believe the behavior is related to incorrect MaxWaitTime related processing in com.sun.genericra.inbound.async.InboundJmsResorcePool. There, in method PauseObject.pauseCallingThread are two errors: First, an elapsed wait time is calculated by

elapsedWaitTime = startTime - System.currentTimeMillis(); // line 415

so this elapsedWaitTime is either 0 or negative.

After that, a remaining wait time is calculated by

remainingWaitTime = startTime - elapsedWaitTime; // line 423

So this value contains roughly the number of milliseconds since 1.1.1970. In line 436, this value is used as input to Object.wait(). So the only way that this wait can terminate (other than waiting ~44 years) is by a call to notify(), which apparently may get lost.

We see in the server's thread dumps, that threads hang exactly in this call to Object.wait() in line 436 after we notice that inbound message processing stops.

The functionality can be fixed by changing line 415 to

elapsedWaitTime = System.currentTimeMillis() - startTime;

and line 426 to

remainingWaitTime = maxWaitTime - elapsedWaitTime;

Then, if the call to notify() gets lost, a JMSException will be thrown in line 420 and message processing will resume.

Environment

SLES 10, GlassFish 2.1.1 p17, JDK 1.5.0_36, MQ Client 7.0.1.6

Affected Versions

[current]

[arinban]

• Issue Imported From: https://github.com/ari-ban/test1/issues/56
• Original Issue Raised By:@arinban
• Original Issue Assigned To: @glassfishrobot

[arinban]

@arinban Commented

• Issue Imported From: https://github.com/ari-ban/glassfish-genericjmsra/issues/56
• Original Issue Raised By:@arinban
• Original Issue Assigned To: @glassfishrobot

[arinban]

@arinban Commented @arinban Commented

• Issue Imported From: https://github.com/javaee/glassfish-genericjmsra/issues/56
• Original Issue Raised By:@glassfishrobot
• Original Issue Assigned To: @glassfishrobot

[arinban]

@arinban Commented @arinban Commented @glassfishrobot Commented Reported by kwinterm

[arinban]

@arinban Commented @arinban Commented @glassfishrobot Commented gerhard_niklasch said: This is being subsumed into #57 (which describes the real cause for the inbound message processing hang).

Note that fixing lines 415 and 423 as per the Description results in the MaxWaitTime default of 3s (unless otherwise specified) being enforced, which had never been the case before. Since 3s is too short for the real world, this risks breaking existing applications – thus the default must also be changed. Changing it to 0, meaning "infinite", will preserve the legacy behavior of MaxWaitTime never being enforced via a JMSException thrown to the driver, and will still allow developers to specify their own preferred MaxWaitTime if desired.

[arinban]

@arinban Commented @arinban Commented @glassfishrobot Commented This issue was imported from java.net JIRA GENERICJMSRA-56

[arinban]

Closing this as this issue is migrated to https://github.com/ari-ban/test1/issues/56