When a JMS listener indicates "no thread available" as in the stack trace below, periodically trigger a thread dump to understand what's tying up workflow threads. The first time the scenario occurs, we should trigger the thread dump. Thereafter, to avoid polluting stdout we should keep a counter, and when the counter reaches a certain config-driven point, trigger another dump and reset the counter. Or maybe we just trigger the first time, and then again when the backup queue is full.
java.lang.Exception: Scheduler has no thread available for Unscheduled event: InternalEvent.262690909startproc203942469 message:
<EventMessage xmlns="http://mdw.centurylink.com/bpm"><EventType>START</EventType>...
at com.centurylink.mdw.services.event.EventServicesImpl.processInternalEvents(EventServicesImpl.java:700) [mdw-services-6.1.31.jar!/:na]
at com.centurylink.mdw.timer.startup.UnscheduledEventMonitor.processUnscheduledEvents(UnscheduledEventMonitor.java:74) [mdw-services-6.1.31.jar!/:na]
at com.centurylink.mdw.timer.startup.UnscheduledEventMonitor.run(UnscheduledEventMonitor.java:60) [mdw-services-6.1.31.jar!/:na]
at java.util.TimerThread.mainLoop(Timer.java:555) [na:1.8.0_222]
at java.util.TimerThread.run(Timer.java:505) [na:1.8.0_222]
When a JMS listener indicates "no thread available" as in the stack trace below, periodically trigger a thread dump to understand what's tying up workflow threads. The first time the scenario occurs, we should trigger the thread dump. Thereafter, to avoid polluting stdout we should keep a counter, and when the counter reaches a certain config-driven point, trigger another dump and reset the counter. Or maybe we just trigger the first time, and then again when the backup queue is full.