Open VinceBezzina opened 9 months ago
jhalterman
commented
9 months ago It looks like the thread is blocked on something that's happening inside your ExpirationListener. Is your expiration listener using some logging code that locks internally, such as on Monitoring.LogMonitor.LogMonitorStream? There may be a deadlock there.
VinceBezzina
commented
9 months ago Its doing a simple System.out.println(), but this is LogMonitorStream - a custom log manager So that is the connection
We stripped out any print statements from the expiringmap listener. But the prints are coming from many different sections of the code - it was the listener that particularly seems to be blocking.
On Fri, Sep 29, 2023 at 5:08 PM Jonathan Halterman @.***> wrote:
It looks like the thread is blocked on something that's happening inside your ExpirationListener. Is your expiration listener using some logging code that locks internally, such as on Monitoring.LogMonitor.LogMonitorStream? There may be a deadlock there.
— Reply to this email directly, view it on GitHub https://github.com/jhalterman/expiringmap/issues/85#issuecomment-1741043463, or unsubscribe https://github.com/notifications/unsubscribe-auth/ALBQ7C6U77QMQFCBLDAUE5DX43P6VANCNFSM6AAAAAA5MR2HTU . You are receiving this because you authored the thread.Message ID: @.***>
jhalterman
commented
9 months ago It seems like something is taking a lock on LogMonitorStream, which leads to the deadlock.
Is it possible that ExpiringMap is not 100% Thread safe.
Name: ExpiringMap-Listener-2322 State: BLOCKED on Monitoring.LogMonitor.LogMonitorStream@108d2eca owned by: ajp-nio-192.168.62.11-8009-exec-11 Total blocked: 1 Total waited: 0
Stack trace: java.io.PrintStream.println(PrintStream.java:805) Vinatra.XXXExpirationListener.expired(XXXExpirationListener.java:41) Vinatra.XXXExpirationListener.expired(XXXExpirationListener.java:15) net.jodah.expiringmap.ExpiringMap$4.run(ExpiringMap.java:1220) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:748)
We had a run off with thousands of threads created and blocking. Here is a sample of the thread. Outside seeing ExpiringMap in the thread. there a system.out.println() in XXXExpirationListener and these system.out.println() are handled by a thread with a custom Log Manager.