search

ClearVolume / clearvolume

ClearVolume is a real-time live 3D visualization library designed for high-end volumetric microscopes such as SPIM and DLSM microscopes. With ClearVolume you can see live on your screen the stacks acquired by your microscope instead of waiting for offline post-processing to give you an intuitive and comprehensive view on your data.
http://clearvolume.github.io
GNU Lesser General Public License v3.0
44 stars 11 forks source link

Can't open large volume #46

Open kthorn opened 8 years ago

kthorn commented 8 years ago

When I try to open a 2560 x 2160 x 197 volume in ClearVolume I get the following error:

java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.RuntimeException: Number of elements in Container too big, use for example CellContainer instead: 2178662400 > 2147483647 at net.imagej.legacy.LegacyService.runLegacyCompatibleCommand(LegacyService.java:277) at net.imagej.legacy.DefaultLegacyHooks.interceptRunPlugIn(DefaultLegacyHooks.java:163) at ij.IJ.runPlugIn(IJ.java) at ij.Executer.runCommand(Executer.java:137) at ij.Executer.run(Executer.java:66) at java.lang.Thread.run(Thread.java:745) Caused by: java.util.concurrent.ExecutionException: java.lang.RuntimeException: Number of elements in Container too big, use for example CellContainer instead: 2178662400 > 2147483647 at java.util.concurrent.FutureTask.report(FutureTask.java:122) at java.util.concurrent.FutureTask.get(FutureTask.java:192) at net.imagej.legacy.LegacyService.runLegacyCompatibleCommand(LegacyService.java:273) ... 5 more Caused by: java.lang.RuntimeException: Number of elements in Container too big, use for example CellContainer instead: 2178662400 > 2147483647 at net.imglib2.img.array.ArrayImgFactory.numEntitiesRangeCheck(ArrayImgFactory.java:71) at net.imglib2.img.array.ArrayImgFactory.createByteInstance(ArrayImgFactory.java:79) at net.imglib2.img.array.ArrayImgFactory.createByteInstance(ArrayImgFactory.java:58) at de.mpicbg.jug.clearvolume.ClearVolumeUnsignedShortType.createSuitableNativeImg(ClearVolumeUnsignedShortType.java:320) at net.imglib2.img.array.ArrayImgFactory.create(ArrayImgFactory.java:63) at de.mpicbg.jug.clearvolume.ImgLib2ClearVolume.makeClearVolumeUnsignedShortTypeCopy(ImgLib2ClearVolume.java:380) at de.mpicbg.jug.clearvolume.ImgLib2ClearVolume.makeClearVolumeUnsignedShortTypeCopies(ImgLib2ClearVolume.java:354) at de.mpicbg.jug.clearvolume.ImgLib2ClearVolume.initRealImgs(ImgLib2ClearVolume.java:446) at de.mpicbg.jug.clearvolume.gui.ClearVolumeManager.run(ClearVolumeManager.java:182) at de.mpicbg.jug.clearvolume.gui.GenericClearVolumeGui.launchClearVolumeManager(GenericClearVolumeGui.java:303) at de.mpicbg.jug.clearvolume.gui.GenericClearVolumeGui.(GenericClearVolumeGui.java:160) at de.mpicbg.jug.plugins.ClearVolumePlugin.run(ClearVolumePlugin.java:86) at org.scijava.command.CommandModule.run(CommandModule.java:205) at org.scijava.module.ModuleRunner.run(ModuleRunner.java:167) at org.scijava.module.ModuleRunner.call(ModuleRunner.java:126) at org.scijava.module.ModuleRunner.call(ModuleRunner.java:65) at org.scijava.thread.DefaultThreadService$2.call(DefaultThreadService.java:191) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)

If I crop the image to a smaller size in X and Y it opens fine.

skalarproduktraum commented 8 years ago

Hi @kthorn and thanks for the report! This issue is already known and we're currently testing a fix. The problem is that Java uses 32bit integers to index data arrays, so the capacity basically ends at 2 Gigabytes (Java does not have unsigned types, so integers cover -2^31 to 2^31).

The fix should be out by the end of next week. If you'd like to try a preliminary version, please let me know.

bmoretti commented 8 years ago

Hi! I'm having this same problem with a 239x2560x2160 stack that's over 2 GB in size. @skalarproduktraum, you said in March 24 that a fix for this issue was going out around that time, so I'm wondering if that actually happened or if it might have been delayed?

Thanks a lot for the support!

royerloic commented 8 years ago

Hi Bruno,

Unfortunately the problem goes very deep and has multiple aspects: i) some limitations of Java make it difficult but not impossible to hadle >2G ii) the GPUs have limited memory and many only support <2G iii) for sizes of 2k x 2k x 2k the rendering would be quite slow even on recent GPUs

ClearVolume 2 is taking time to developp, it will be built on top of Scenery (@skalarproduktraum https://github.com/skalarproduktraum) and will not have these limitations. Right now only @skalarproduktraum https://github.com/skalarproduktraum has time to work on this, I just got a GL offer and my pipeline is full, and so I have to wait until I can get a student to work on it to help Ulrik. Or spend two weeks hacking madly… but when ? :-(

In the meantime, Here is what I suggest: downscale your data by a factor 2 in each direction to get a stack of ~ 1k x 1k x1k.

Think about it, your screen has probably about 1000x2000 pixels, so you cannot even see 2000 pixels across all directions on your screen anyway. Moreover, your PSF probably is about 3x3x3 pixels so that also limits the utility of having so many pixels.

A volume of 1000 x 1000 x 1000 is already a beautiful stack that will render very well on ClearVolume if you have a very good graphics card (< 2 years) with at least 4 gigas of GPU RAM. There will be no perceptual loss of quality, because of the limitations induced by the PSF and screen resolution. Yet, you will need 8 times less Video RAM to display this and it will be 8 times faster…

We should probably add this automatic downscaling…

Hope this helps…

Cheers :-)

loic

On 07 Sep 2016, at 22:01, Bruno Moretti notifications@github.com wrote:

Hi! I'm having this same problem with a 239x2560x2160 stack that's over 2 GB in size. @skalarproduktraum https://github.com/skalarproduktraum, you said in March 24 that a fix for this issue was going out around that time, so I'm wondering if that actually happened or if it might have been delayed?

Thanks a lot for the support!

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/ClearVolume/ClearVolume/issues/46#issuecomment-245398981, or mute the thread https://github.com/notifications/unsubscribe-auth/AByMksnjgwB7gHXg7wOVsgmGPr76eFoLks5qnxgqgaJpZM4H3VgV.

Dr. Loïc Alain Royer

Post-Doc - Myers Lab Max Planck Institute of Molecular Cell Biology and Genetics Pfotenhauerstr. 108 01307 Dresden

Too short? Here's why: http://emailcharter.org/