fiji / Trainable_Segmentation

Fiji library to perform image segmentation based on the Weka learning schemes
https://imagej.net/Trainable_Weka_Segmentation
GNU General Public License v3.0
106 stars 60 forks source link

Resource leak: threads left after applying classifier #82

Open imagejan opened 4 months ago

imagejan commented 4 months ago

When I programmatically apply a classifier to an image, not all threads seem to be closed after processing.

This issue hit me when trying to use WekaSegmentation via pyimagej from Python, where the Python process hangs indefinitely after processing. See https://github.com/fmi-faim/faim-wako-searchfirst/pull/57 and this example run for a test that passes, but the GitHub Actions job needed to cancelled because it was hanging forever. I tried calling scyjava.shutdown_jvm() after classifying, but that made it hang before showing the pytest output.

Consider the following Groovy script:

#@ ImagePlus imp
#@ File classifier

import trainableSegmentation.WekaSegmentation

ws = new WekaSegmentation(imp)
ws.loadClassifier(classifier.getAbsolutePath())
ws.applyClassifier(false)
result = ws.getClassifiedImage()

After running the script on the current open image, I see many new threads in Plugins > Utilities > Threads:

        Thread: pool-6-thread-2  Priority: 5
        Thread: pool-6-thread-1  Priority: 5
        Thread: pool-6-thread-5  Priority: 5
        Thread: pool-6-thread-6  Priority: 5
        Thread: pool-6-thread-7  Priority: 5
        Thread: pool-6-thread-4  Priority: 5
        Thread: pool-6-thread-13  Priority: 5
        Thread: pool-6-thread-8  Priority: 5
        Thread: pool-6-thread-16  Priority: 5
        Thread: pool-6-thread-17  Priority: 5
        Thread: pool-6-thread-18  Priority: 5
        Thread: pool-6-thread-19  Priority: 5
        Thread: pool-6-thread-20  Priority: 5
        Thread: pool-6-thread-21  Priority: 5
        Thread: pool-6-thread-22  Priority: 5
        Thread: pool-6-thread-23  Priority: 5
        Thread: pool-6-thread-24  Priority: 5
        Thread: pool-6-thread-14  Priority: 5
        Thread: pool-6-thread-27  Priority: 5
        Thread: pool-6-thread-28  Priority: 5
        Thread: pool-6-thread-29  Priority: 5
        Thread: pool-6-thread-30  Priority: 5
        Thread: pool-6-thread-31  Priority: 5
        Thread: pool-6-thread-32  Priority: 5
        Thread: pool-6-thread-33  Priority: 5
        Thread: pool-6-thread-34  Priority: 5
        Thread: pool-6-thread-35  Priority: 5
        Thread: pool-6-thread-3  Priority: 5
        Thread: pool-6-thread-26  Priority: 5
        Thread: pool-6-thread-15  Priority: 5
        Thread: pool-6-thread-37  Priority: 5
        Thread: pool-6-thread-10  Priority: 5
        Thread: pool-6-thread-40  Priority: 5
        Thread: pool-6-thread-41  Priority: 5
        Thread: pool-6-thread-42  Priority: 5
        Thread: pool-6-thread-43  Priority: 5
        Thread: pool-6-thread-44  Priority: 5
        Thread: pool-6-thread-45  Priority: 5
        Thread: pool-6-thread-46  Priority: 5
        Thread: pool-6-thread-47  Priority: 5
        Thread: pool-6-thread-48  Priority: 5
        Thread: pool-6-thread-49  Priority: 5
        Thread: pool-6-thread-50  Priority: 5
        Thread: pool-6-thread-11  Priority: 5
        Thread: pool-6-thread-39  Priority: 5
        Thread: pool-6-thread-51  Priority: 5
        Thread: pool-6-thread-52  Priority: 5
        Thread: pool-6-thread-38  Priority: 5
        Thread: pool-6-thread-53  Priority: 5
        Thread: pool-6-thread-54  Priority: 5
        Thread: pool-6-thread-55  Priority: 5
        Thread: pool-6-thread-56  Priority: 5
        Thread: pool-6-thread-57  Priority: 5
        Thread: pool-6-thread-58  Priority: 5
        Thread: pool-6-thread-59  Priority: 5
        Thread: pool-6-thread-36  Priority: 5
        Thread: pool-6-thread-60  Priority: 5
        Thread: pool-6-thread-61  Priority: 5
        Thread: pool-6-thread-62  Priority: 5
        Thread: pool-6-thread-63  Priority: 5
        Thread: pool-6-thread-64  Priority: 5
        Thread: pool-6-thread-65  Priority: 5
        Thread: pool-6-thread-9  Priority: 5
        Thread: pool-6-thread-12  Priority: 5
        Thread: pool-6-thread-25  Priority: 5

After a couple of minutes, some of these are still hanging around:

        Thread: pool-6-thread-4  Priority: 5
        Thread: pool-6-thread-13  Priority: 5
        Thread: pool-6-thread-17  Priority: 5
        Thread: pool-6-thread-21  Priority: 5
        Thread: pool-6-thread-22  Priority: 5
        Thread: pool-6-thread-32  Priority: 5
        Thread: pool-6-thread-10  Priority: 5
        Thread: pool-6-thread-47  Priority: 5
        Thread: pool-6-thread-38  Priority: 5
        Thread: pool-6-thread-56  Priority: 5
        Thread: pool-6-thread-36  Priority: 5
        Thread: pool-6-thread-62  Priority: 5

After a second call of the script:

        Thread: pool-6-thread-4  Priority: 5
        Thread: pool-6-thread-13  Priority: 5
        Thread: pool-6-thread-17  Priority: 5
        Thread: pool-6-thread-21  Priority: 5
        Thread: pool-6-thread-22  Priority: 5
        Thread: pool-6-thread-32  Priority: 5
        Thread: pool-6-thread-10  Priority: 5
        Thread: pool-6-thread-47  Priority: 5
        Thread: pool-6-thread-38  Priority: 5
        Thread: pool-6-thread-56  Priority: 5
        Thread: pool-6-thread-36  Priority: 5
        Thread: pool-6-thread-62  Priority: 5
        Thread: pool-9-thread-1  Priority: 5
        Thread: pool-10-thread-1  Priority: 5
        Thread: pool-6-thread-66  Priority: 5
        Thread: pool-6-thread-67  Priority: 5
        Thread: pool-6-thread-68  Priority: 5
        Thread: pool-6-thread-69  Priority: 5
        Thread: pool-6-thread-71  Priority: 5
        Thread: pool-6-thread-72  Priority: 5
        Thread: pool-6-thread-73  Priority: 5
        Thread: pool-6-thread-74  Priority: 5
        Thread: pool-6-thread-76  Priority: 5
        Thread: pool-6-thread-77  Priority: 5
        Thread: pool-6-thread-79  Priority: 5
        Thread: pool-6-thread-81  Priority: 5
        Thread: pool-6-thread-82  Priority: 5
        Thread: pool-6-thread-83  Priority: 5
        Thread: pool-6-thread-84  Priority: 5
        Thread: pool-6-thread-70  Priority: 5
        Thread: pool-6-thread-80  Priority: 5
        Thread: pool-6-thread-75  Priority: 5
        Thread: pool-6-thread-78  Priority: 5
        Thread: pool-6-thread-87  Priority: 5
        Thread: pool-6-thread-88  Priority: 5
        Thread: pool-6-thread-89  Priority: 5
        Thread: pool-6-thread-90  Priority: 5
        Thread: pool-6-thread-92  Priority: 5
        Thread: pool-6-thread-93  Priority: 5
        Thread: pool-6-thread-94  Priority: 5
        Thread: pool-6-thread-95  Priority: 5
        Thread: pool-6-thread-96  Priority: 5
        Thread: pool-6-thread-97  Priority: 5
        Thread: pool-6-thread-98  Priority: 5
        Thread: pool-6-thread-86  Priority: 5
        Thread: pool-6-thread-85  Priority: 5
        Thread: pool-6-thread-99  Priority: 5
        Thread: pool-6-thread-101  Priority: 5
        Thread: pool-6-thread-102  Priority: 5
        Thread: pool-6-thread-91  Priority: 5
        Thread: pool-6-thread-103  Priority: 5
        Thread: pool-6-thread-104  Priority: 5
        Thread: pool-6-thread-105  Priority: 5
        Thread: pool-6-thread-106  Priority: 5
        Thread: pool-6-thread-107  Priority: 5
        Thread: pool-6-thread-100  Priority: 5
        Thread: pool-6-thread-108  Priority: 5
        Thread: pool-6-thread-109  Priority: 5
        Thread: pool-6-thread-110  Priority: 5
        Thread: pool-6-thread-111  Priority: 5
        Thread: pool-6-thread-112  Priority: 5
        Thread: pool-6-thread-113  Priority: 5
        Thread: pool-6-thread-114  Priority: 5
        Thread: pool-6-thread-115  Priority: 5

And again, a couple of minutes later:

        Thread: pool-6-thread-4  Priority: 5
        Thread: pool-6-thread-17  Priority: 5
        Thread: pool-6-thread-38  Priority: 5
        Thread: pool-6-thread-67  Priority: 5
        Thread: pool-6-thread-96  Priority: 5
        Thread: pool-6-thread-97  Priority: 5
        Thread: pool-6-thread-85  Priority: 5
        Thread: pool-6-thread-99  Priority: 5
        Thread: pool-6-thread-104  Priority: 5
        Thread: pool-6-thread-100  Priority: 5
        Thread: pool-6-thread-113  Priority: 5
        Thread: pool-6-thread-114  Priority: 5

It seems a few threads are quite persistent.