Java thread pool leak bug in idp-plugin-privacyIDEA-1.1.0

[equbaykiflay]

I opened this bug on Shibboleth but it was found that the bug is actually in the privacyIDEA plugin. I verified that by disabling the plugin and checking if the problem disappears, which it did. So I am posting it here

Description

Thread pools are not being released resulting in thread and memory leak. Here is the output of jstack with a threads aged over 11 hours and less than 10 second stuck waiting on condition.

root@shibserver:~# jstack 1115417 | grep -A 14 thread-1 | head -30 "pool-3-thread-1" #105 prio=5 os_prio=0 cpu=36.14ms elapsed=42650.98s tid=0x00007f52f819fe80 nid=0x1106a7 waiting on condition [0x00007f52d6ffe000] java.lang.Thread.State: WAITING (parking) at jdk.internal.misc.Unsafe.park(java.base@17.0.12/Native Method)

• parking to wait for <0x00000005aeab6258> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(java.base@17.0.12/LockSupport.java:341) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionNode.block(java.base@17.0.12/AbstractQueuedSynchronizer.java:506) at java.util.concurrent.ForkJoinPool.unmanagedBlock(java.base@17.0.12/ForkJoinPool.java:3465) at java.util.concurrent.ForkJoinPool.managedBlock(java.base@17.0.12/ForkJoinPool.java:3436) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(java.base@17.0.12/AbstractQueuedSynchronizer.java:1625) at java.util.concurrent.ArrayBlockingQueue.take(java.base@17.0.12/ArrayBlockingQueue.java:420) at java.util.concurrent.ThreadPoolExecutor.getTask(java.base@17.0.12/ThreadPoolExecutor.java:1062) at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@17.0.12/ThreadPoolExecutor.java:1122) at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@17.0.12/ThreadPoolExecutor.java:635) at java.lang.Thread.run(java.base@17.0.12/Thread.java:840)

"pool-4-thread-1" #109 prio=5 os_prio=0 cpu=0.62ms elapsed=42649.61s tid=0x00007f52f81a2ff0 nid=0x1106af waiting on condition [0x00007f52d69f8000] java.lang.Thread.State: WAITING (parking) at jdk.internal.misc.Unsafe.park(java.base@17.0.12/Native Method)

• parking to wait for <0x00000005aeab64b8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(java.base@17.0.12/LockSupport.java:341) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionNode.block(java.base@17.0.12/AbstractQueuedSynchronizer.java:506) at java.util.concurrent.ForkJoinPool.unmanagedBlock(java.base@17.0.12/ForkJoinPool.java:3465) at java.util.concurrent.ForkJoinPool.managedBlock(java.base@17.0.12/ForkJoinPool.java:3436) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(java.base@17.0.12/AbstractQueuedSynchronizer.java:1625) at java.util.concurrent.ArrayBlockingQueue.take(java.base@17.0.12/ArrayBlockingQueue.java:420) at java.util.concurrent.ThreadPoolExecutor.getTask(java.base@17.0.12/ThreadPoolExecutor.java:1062) at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@17.0.12/ThreadPoolExecutor.java:1122) at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@17.0.12/ThreadPoolExecutor.java:635) at java.lang.Thread.run(java.base@17.0.12/Thread.java:840)

root@shibserver:~# jstack 1115417 | grep -A 14 thread-1 | tail -32 "pool-6929-thread-1" #32245 prio=5 os_prio=0 cpu=0.57ms elapsed=7.75s tid=0x00007f535e314060 nid=0x122c78 waiting on condition [0x00007f5122ad4000] java.lang.Thread.State: WAITING (parking) at jdk.internal.misc.Unsafe.park(java.base@17.0.12/Native Method)

• parking to wait for <0x00000004acb9a2d8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(java.base@17.0.12/LockSupport.java:341) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionNode.block(java.base@17.0.12/AbstractQueuedSynchronizer.java:506) at java.util.concurrent.ForkJoinPool.unmanagedBlock(java.base@17.0.12/ForkJoinPool.java:3465) at java.util.concurrent.ForkJoinPool.managedBlock(java.base@17.0.12/ForkJoinPool.java:3436) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(java.base@17.0.12/AbstractQueuedSynchronizer.java:1625) at java.util.concurrent.ArrayBlockingQueue.take(java.base@17.0.12/ArrayBlockingQueue.java:420) at java.util.concurrent.ThreadPoolExecutor.getTask(java.base@17.0.12/ThreadPoolExecutor.java:1062) at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@17.0.12/ThreadPoolExecutor.java:1122) at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@17.0.12/ThreadPoolExecutor.java:635) at java.lang.Thread.run(java.base@17.0.12/Thread.java:840)

-- "pool-6930-thread-1" #32248 prio=5 os_prio=0 cpu=0.21ms elapsed=6.41s tid=0x00007f52f4724470 nid=0x122c7b waiting on condition [0x00007f51249f3000] java.lang.Thread.State: WAITING (parking) at jdk.internal.misc.Unsafe.park(java.base@17.0.12/Native Method)

• parking to wait for <0x00000004acdb1000> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(java.base@17.0.12/LockSupport.java:341) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionNode.block(java.base@17.0.12/AbstractQueuedSynchronizer.java:506) at java.util.concurrent.ForkJoinPool.unmanagedBlock(java.base@17.0.12/ForkJoinPool.java:3465) at java.util.concurrent.ForkJoinPool.managedBlock(java.base@17.0.12/ForkJoinPool.java:3436) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(java.base@17.0.12/AbstractQueuedSynchronizer.java:1625) at java.util.concurrent.ArrayBlockingQueue.take(java.base@17.0.12/ArrayBlockingQueue.java:420) at java.util.concurrent.ThreadPoolExecutor.getTask(java.base@17.0.12/ThreadPoolExecutor.java:1062) at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@17.0.12/ThreadPoolExecutor.java:1122) at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@17.0.12/ThreadPoolExecutor.java:635) at java.lang.Thread.run(java.base@17.0.12/Thread.java:840)

Environment

OS: Debian 12.7 Container: Jetty 12.0.13 JVM: amazon-corretto 17.0.12.7.1 (build 17.0.12+7-LTS, mixed mode, sharing) Idp: Shibboleth 5.1.3 PI plugin: idp-plugin-privacyIDEA 1.1.0

[nilsbehlen]

@lukasmatusiewicz does this plugin in v1.1.0 use the javaclient in v1.2.2? I have made changes to the java client to fix this problem. it was reported previously with keycloak. If not, we need to build a new version of this plugin with the current javaclient.

[lukasmatusiewicz]

@nilsbehlen it uses v1.2.0 - 17 Jan 2023, so we need to update it