search

fiji / Directionality

A Fiji plugin for inferring the main orientation of structures in an image.
https://imagej.net/plugins/directionality
5 stars 5 forks source link

RAM consumption for large images #4

Closed Ender117 closed 2 years ago

Ender117 commented 2 years ago

When processing large images (e.g., stitched SEM of ~9500*4500 px) using Fourier components method, RAM consumption was >20G after 5min+ and does not finish. Is this expected, or am I looking at memory leak?

>>>>>>>>>>>>>>>>>>>>>>>>>>>
<Out of memory>
<All available memory (24436MB) has been>
<used. To make more available, use the>
<Edit>Options>Memory & Threads command.>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
ctrueden commented 2 years ago

@Ender117 I'm not sure. @tinevez What do you think?

One thing you can do when an operation hangs for a long time is press shift+\ to run the Stack Dump plugin. It will open a window with full stack trace of every Java thread. Sometimes that information can be helpful for seeing where things are stuck. But it's not as informative as using a profiler like Visual VM; see Debugging memory leaks on the ImageJ wiki for a brief guide on that.

Ender117 commented 2 years ago 
[Mon May 02 22:22:31 EDT 2022] [ERROR] [] Exception during event handling:
    [Event] org.scijava.display.event.DisplayActivatedEvent
    context = org.scijava.Context@ed3068a
    consumed = false
    display = plugin:org.scijava.display.DefaultTextDisplay: type=class java.lang.String, name=Stack Dump, objects={"AWT-EventQueue-0" prio=6 id=18 group=main
   java.lang.Thread.State: WAITING
    at sun.misc.Unsafe.park(Native Method)
    at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
    at java.awt.EventQueue.getNextEvent(EventQueue.java:554)
    at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:187)
    at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:116)
    at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:105)
    at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101)
    at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:93)
    at java.awt.EventDispatchThread.run(EventDispatchThread.java:82)

"AWT-Shutdown" prio=5 id=15 group=system
   java.lang.Thread.State: WAITING
    at java.lang.Object.wait(Native Method)
    at java.lang.Object.wait(Object.java:502)
    at sun.awt.AWTAutoShutdown.run(AWTAutoShutdown.java:295)
    at java.lang.Thread.run(Thread.java:748)

"AWT-Windows" daemon prio=6 id=16 group=system
   java.lang.Thread.State: RUNNABLE
    at sun.awt.windows.WToolkit.eventLoop(Native Method)
    at sun.awt.windows.WToolkit.run(WToolkit.java:312)
    at java.lang.Thread.run(Thread.java:748)

"Attach Listener" daemon prio=5 id=6 group=system
   java.lang.Thread.State: RUNNABLE

"D3D Screen Updater" daemon prio=7 id=24 group=system
   java.lang.Thread.State: TIMED_WAITING
    at java.lang.Object.wait(Native Method)
    at sun.java2d.d3d.D3DScreenUpdateManager.run(D3DScreenUpdateManager.java:423)
    at java.lang.Thread.run(Thread.java:748)

"DestroyJavaVM" prio=5 id=25 group=main
   java.lang.Thread.State: RUNNABLE

"Directionality" prio=4 id=36 group=main
   java.lang.Thread.State: RUNNABLE
    at fiji.analyze.directionality.Directionality_.makeFftFilters(Directionality_.java:1728)
    at fiji.analyze.directionality.Directionality_.initFourierFields(Directionality_.java:1313)
    at fiji.analyze.directionality.Directionality_.computeHistograms(Directionality_.java:566)
    at fiji.analyze.directionality.Directionality_.run(Directionality_.java:497)
    at ij.IJ.runUserPlugIn(IJ.java:241)
    at ij.IJ.runPlugIn(IJ.java:204)
    at ij.Executer.runCommand(Executer.java:150)
    at ij.Executer.run(Executer.java:68)
    at java.lang.Thread.run(Thread.java:748)

"Finalizer" daemon prio=8 id=3 group=system
   java.lang.Thread.State: WAITING
    at java.lang.Object.wait(Native Method)
    at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:144)
    at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:165)
    at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:216)

"Java2D Disposer" daemon prio=10 id=14 group=system
   java.lang.Thread.State: WAITING
    at java.lang.Object.wait(Native Method)
    at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:144)
    at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:165)
    at sun.java2d.Disposer.run(Disposer.java:148)
    at java.lang.Thread.run(Thread.java:748)

"Reference Handler" daemon prio=10 id=2 group=system
   java.lang.Thread.State: WAITING
    at java.lang.Object.wait(Native Method)
    at java.lang.Object.wait(Object.java:502)
    at java.lang.ref.Reference.tryHandlePending(Reference.java:191)
    at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:153)

"SciJava-ed3068a-Thread-2" prio=6 id=37 group=main
   java.lang.Thread.State: RUNNABLE
    at java.lang.Thread.dumpThreads(Native Method)
    at java.lang.Thread.getAllStackTraces(Thread.java:1610)
    at org.scijava.plugins.commands.debug.DumpStack.run(DumpStack.java:81)
    at org.scijava.command.CommandModule.run(CommandModule.java:196)
    at org.scijava.module.ModuleRunner.run(ModuleRunner.java:163)
    at org.scijava.module.ModuleRunner.call(ModuleRunner.java:124)
    at org.scijava.module.ModuleRunner.call(ModuleRunner.java:63)
    at org.scijava.thread.DefaultThreadService.lambda$wrap$2(DefaultThreadService.java:225)
    at org.scijava.thread.DefaultThreadService$$Lambda$94/44983976.call(Unknown Source)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)

"SciJava-ed3068a-Thread-3" prio=6 id=38 group=main
   java.lang.Thread.State: TIMED_WAITING
    at sun.misc.Unsafe.park(Native Method)
    at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
    at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
    at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
    at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
    at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)

"Signal Dispatcher" daemon prio=9 id=5 group=system
   java.lang.Thread.State: RUNNABLE

}
    [Subscriber] net.imagej.legacy.LegacyService [priority = 1.0]
    [Method] private void net.imagej.legacy.LegacyService.onEvent(org.scijava.display.event.DisplayActivatedEvent)
java.lang.AbstractMethodError: Method net/imglib2/img/VirtualStackAdapter$ImagePlusLoader$$Lambda$102.extract(Ljava/lang/Object;)Ljava/lang/Object; is abstract
    at net.imglib2.img.VirtualStackAdapter$ImagePlusLoader$$Lambda$102/336410412.extract(Unknown Source)
    at net.imglib2.cache.ref.SoftRefLoaderRemoverCache$Entry.setValue(SoftRefLoaderRemoverCache.java:115)
    at net.imglib2.cache.ref.SoftRefLoaderRemoverCache.get(SoftRefLoaderRemoverCache.java:167)
    at net.imglib2.cache.util.LoaderRemoverCacheAsRemoverCacheAdapter.get(LoaderRemoverCacheAsRemoverCacheAdapter.java:78)
    at net.imglib2.cache.util.RemoverCacheAsCacheAdapter.get(RemoverCacheAsCacheAdapter.java:76)
    at net.imglib2.img.VirtualStackAdapter$ImagePlusLoader.get(VirtualStackAdapter.java:178)
    at net.imglib2.img.VirtualStackAdapter$ImagePlusLoader.get(VirtualStackAdapter.java:158)
    at net.imglib2.img.planar.PlanarImg.update(PlanarImg.java:118)
    at net.imglib2.img.planar.PlanarImg.update(PlanarImg.java:66)
    at net.imglib2.type.numeric.integer.GenericByteType.updateContainer(GenericByteType.java:100)
    at net.imglib2.img.planar.PlanarCursor.reset(PlanarCursor.java:167)
    at net.imglib2.img.planar.PlanarCursor.<init>(PlanarCursor.java:95)
    at net.imglib2.img.planar.PlanarCursor2D.<init>(PlanarCursor2D.java:50)
    at net.imglib2.img.planar.PlanarImg.cursor(PlanarImg.java:217)
    at net.imglib2.img.planar.PlanarImg.cursor(PlanarImg.java:66)
    at net.imglib2.img.AbstractImg.firstElement(AbstractImg.java:81)
    at net.imagej.ImgPlus.firstElement(ImgPlus.java:274)
    at net.imagej.DefaultDatasetService.create(DefaultDatasetService.java:206)
    at net.imagej.legacy.translate.DisplayCreator.makeDataset(DisplayCreator.java:139)
    at net.imagej.legacy.translate.DisplayCreator.getDataset(DisplayCreator.java:101)
    at net.imagej.legacy.translate.DisplayCreator.makeDisplay(DisplayCreator.java:179)
    at net.imagej.legacy.translate.DisplayCreator.createDisplay(DisplayCreator.java:93)
    at net.imagej.legacy.translate.ImageTranslator.createDisplay(ImageTranslator.java:64)
    at net.imagej.legacy.LegacyImageMap.registerLegacyImage(LegacyImageMap.java:288)
    at net.imagej.legacy.display.LegacyImageDisplayService.getActiveImageDisplay(LegacyImageDisplayService.java:117)
    at net.imagej.legacy.LegacyService.syncActiveImage(LegacyService.java:318)
    at net.imagej.legacy.LegacyService.onEvent(LegacyService.java:563)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.scijava.event.DefaultEventService$ProxySubscriber.onEvent(DefaultEventService.java:298)
    at org.scijava.event.DefaultEventService$ProxySubscriber.onEvent(DefaultEventService.java:272)
    at org.bushe.swing.event.ThreadSafeEventService.publish(ThreadSafeEventService.java:971)
    at org.scijava.event.DefaultEventBus.access$101(DefaultEventBus.java:54)
    at org.scijava.event.DefaultEventBus$1.run(DefaultEventBus.java:188)
    at org.scijava.thread.DefaultThreadService.invoke(DefaultThreadService.java:111)
    at org.scijava.event.DefaultEventBus.publishNow(DefaultEventBus.java:182)
    at org.scijava.event.DefaultEventBus.publishNow(DefaultEventBus.java:73)
    at org.scijava.event.DefaultEventService.publish(DefaultEventService.java:102)
    at org.scijava.display.DefaultDisplayService.setActiveDisplay(DefaultDisplayService.java:123)
    at org.scijava.display.DefaultDisplayService.onEvent(DefaultDisplayService.java:248)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.scijava.event.DefaultEventService$ProxySubscriber.onEvent(DefaultEventService.java:298)
    at org.scijava.event.DefaultEventService$ProxySubscriber.onEvent(DefaultEventService.java:272)
    at org.bushe.swing.event.ThreadSafeEventService.publish(ThreadSafeEventService.java:971)
    at org.scijava.event.DefaultEventBus.access$101(DefaultEventBus.java:54)
    at org.scijava.event.DefaultEventBus$1.run(DefaultEventBus.java:188)
    at org.scijava.thread.DefaultThreadService.invoke(DefaultThreadService.java:111)
    at org.scijava.event.DefaultEventBus.publishNow(DefaultEventBus.java:182)
    at org.scijava.event.DefaultEventBus.publishNow(DefaultEventBus.java:73)
    at org.scijava.event.DefaultEventService.publish(DefaultEventService.java:102)
    at org.scijava.ui.awt.AWTWindowEventDispatcher.windowActivated(AWTWindowEventDispatcher.java:90)
    at java.awt.Window.processWindowEvent(Window.java:2066)
    at javax.swing.JFrame.processWindowEvent(JFrame.java:305)
    at java.awt.Window.processEvent(Window.java:2013)
    at java.awt.Component.dispatchEventImpl(Component.java:4889)
    at java.awt.Container.dispatchEventImpl(Container.java:2297)
    at java.awt.Window.dispatchEventImpl(Window.java:2746)
    at java.awt.Component.dispatchEvent(Component.java:4711)
    at java.awt.KeyboardFocusManager.redispatchEvent(KeyboardFocusManager.java:1954)
    at java.awt.DefaultKeyboardFocusManager.typeAheadAssertions(DefaultKeyboardFocusManager.java:1024)
    at java.awt.DefaultKeyboardFocusManager.dispatchEvent(DefaultKeyboardFocusManager.java:516)
    at java.awt.Component.dispatchEventImpl(Component.java:4760)
    at java.awt.Container.dispatchEventImpl(Container.java:2297)
    at java.awt.Window.dispatchEventImpl(Window.java:2746)
    at java.awt.Component.dispatchEvent(Component.java:4711)
    at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:760)
    at java.awt.EventQueue.access$500(EventQueue.java:97)
    at java.awt.EventQueue$3.run(EventQueue.java:709)
    at java.awt.EventQueue$3.run(EventQueue.java:703)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:74)
    at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:84)
    at java.awt.EventQueue$4.run(EventQueue.java:733)
    at java.awt.EventQueue$4.run(EventQueue.java:731)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:74)
    at java.awt.EventQueue.dispatchEvent(EventQueue.java:730)
    at java.awt.SentEvent.dispatch(SentEvent.java:70)
    at java.awt.DefaultKeyboardFocusManager$DefaultKeyboardFocusManagerSentEvent.dispatch(DefaultKeyboardFocusManager.java:239)
    at java.awt.DefaultKeyboardFocusManager.sendMessage(DefaultKeyboardFocusManager.java:266)
    at java.awt.DefaultKeyboardFocusManager.dispatchEvent(DefaultKeyboardFocusManager.java:388)
    at java.awt.Component.dispatchEventImpl(Component.java:4760)
    at java.awt.Container.dispatchEventImpl(Container.java:2297)
    at java.awt.Window.dispatchEventImpl(Window.java:2746)
    at java.awt.Component.dispatchEvent(Component.java:4711)
    at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:760)
    at java.awt.EventQueue.access$500(EventQueue.java:97)
    at java.awt.EventQueue$3.run(EventQueue.java:709)
    at java.awt.EventQueue$3.run(EventQueue.java:703)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:74)
    at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:84)
    at java.awt.EventQueue$4.run(EventQueue.java:733)
    at java.awt.EventQueue$4.run(EventQueue.java:731)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:74)
    at java.awt.EventQueue.dispatchEvent(EventQueue.java:730)
    at java.awt.SequencedEvent.dispatch(SequencedEvent.java:128)
    at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:758)
    at java.awt.EventQueue.access$500(EventQueue.java:97)
    at java.awt.EventQueue$3.run(EventQueue.java:709)
    at java.awt.EventQueue$3.run(EventQueue.java:703)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:74)
    at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:84)
    at java.awt.EventQueue$4.run(EventQueue.java:733)
    at java.awt.EventQueue$4.run(EventQueue.java:731)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:74)
    at java.awt.EventQueue.dispatchEvent(EventQueue.java:730)
    at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:205)
    at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:116)
    at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:105)
    at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101)
    at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:93)
    at java.awt.EventDispatchThread.run(EventDispatchThread.java:82)

This is the output of the suggested dump. Hope this helps.

ctrueden commented 2 years ago

@Ender117 Two things from the stack trace jump out at me:

  1. What the Directionality plugin is doing:

    at fiji.analyze.directionality.Directionality_.makeFftFilters(Directionality_.java:1728)
at fiji.analyze.directionality.Directionality_.initFourierFields(Directionality_.java:1313)
at fiji.analyze.directionality.Directionality_.computeHistograms(Directionality_.java:566)
at fiji.analyze.directionality.Directionality_.run(Directionality_.java:497)

  2. The AbstractMethodError:

    java.lang.AbstractMethodError: Method net/imglib2/img/VirtualStackAdapter$ImagePlusLoader$$Lambda$102.extract(Ljava/lang/Object;)Ljava/lang/Object; is **abstract**

The Directionality plugin stack trace (1) is here in the code. If I'm reading the code correctly, the pad_size is computed as 8192 (the smallest power of 2 larger than the smaller image dimension for 9500*4500). The default nbins is 90 (are you changing it, @Ender117?). The plugin creates an image stack dimensioned:

final ImageStack filters = new ImageStack( pad_size, pad_size, nbins );

So in this case, it will be 90 planes of 64Mpix each, using 4-byte floats, so 256MB*90 = 23GB.

So your observation that it takes >20GB is in line with the algorithm. Unfortunately, that's just how much RAM it uses to make the FFT filters image stack. You could reduce the memory requirement by either: A) using a smaller nbins; or B) downsampling your image.

ctrueden commented 2 years ago

Oh, and regarding the AbstractMethodError: that is not expected, and indicates that perhaps there is a version skew in your Fiji's JAR files. Are you using a full updated Fiji with no extra update sites enabled?