search

saalfeldlab / paintera

GNU General Public License v2.0
94 stars 17 forks source link

Unable to create new Dataset #497

Closed ddenberg closed 11 months ago

ddenberg commented 11 months ago

Hi! I'm interested in using Paintera to manually label a dataset I'm working with, however I'm running into an issue with creating a label source. I am using windows 10, and I installed Paintera via conda.

I've successfully loaded my raw data (hdf5 format). Because I'm not working from an n5 data set, I set the following in Sources > New > Label Source:

Name: temp
N5: C:\path to empty directory for labels\
Dataset: temp
Dimensions: 2048, 2048, 101
Block Size: 2048, 2048, 101
Resolution: 0.208, 0.208, 2.0
Offset: 0.0, 0.0, 0.0

I get the following pop up error after selecting "Create": image

Also in the console the following warnings appear:

WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.janelia.saalfeldlab.n5.ReflectionUtils (file:/C:/Users/ddenberg/.jgo/org.janelia.saalfeldlab/paintera/0.34.1/147e97a0f71f03de2cd2423a831a1e84571bf38b542f9392fd56fb2fda35bf64/n5-2.5.1.jar) to field java.lang.reflect.Field.modifiers
WARNING: Please consider reporting this to the maintainers of org.janelia.saalfeldlab.n5.ReflectionUtils
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release

Also, here is the stack trace from the pop-up error if that helps:

java.nio.BufferOverflowException
    at java.base/java.nio.Buffer.nextPutIndex(Buffer.java:673)
    at java.base/java.nio.HeapByteBuffer.putLong(HeapByteBuffer.java:463)
    at org.janelia.saalfeldlab.n5.imglib2.N5LabelMultisetCacheLoader$ConstantNullReplacement.apply(N5LabelMultisetCacheLoader.java:180)
    at org.janelia.saalfeldlab.n5.imglib2.N5LabelMultisetCacheLoader$ConstantNullReplacement.apply(N5LabelMultisetCacheLoader.java:106)
    at org.janelia.saalfeldlab.n5.imglib2.N5LabelMultisetCacheLoader.getData(N5LabelMultisetCacheLoader.java:103)
    at net.imglib2.type.label.AbstractLabelMultisetLoader.get(AbstractLabelMultisetLoader.java:60)
    at net.imglib2.type.label.AbstractLabelMultisetLoader.get(AbstractLabelMultisetLoader.java:21)
    at net.imglib2.cache.ref.SoftRefLoaderCache.get(SoftRefLoaderCache.java:135)
    at net.imglib2.cache.util.LoaderCacheAsCacheAdapter.get(LoaderCacheAsCacheAdapter.java:76)
    at net.imglib2.cache.util.CacheAsUncheckedCacheAdapter.get(CacheAsUncheckedCacheAdapter.java:74)
    at net.imglib2.img.cell.LazyCellImg$LazyCells.get(LazyCellImg.java:105)
    at net.imglib2.img.list.AbstractLongListImg$LongListRandomAccess.get(AbstractLongListImg.java:274)
    at net.imglib2.img.cell.CellRandomAccess.getCell(CellRandomAccess.java:142)
    at net.imglib2.img.cell.CellRandomAccess.updatePosition(CellRandomAccess.java:474)
    at net.imglib2.img.cell.CellRandomAccess.<init>(CellRandomAccess.java:136)
    at net.imglib2.img.cell.AbstractCellImg.randomAccess(AbstractCellImg.java:105)
    at net.imglib2.img.cell.AbstractCellImg.randomAccess(AbstractCellImg.java:52)
    at net.imglib2.util.Util.getTypeFromInterval(Util.java:806)
    at org.janelia.saalfeldlab.paintera.data.RandomAccessibleIntervalDataSource.<init>(RandomAccessibleIntervalDataSource.java:134)
    at org.janelia.saalfeldlab.paintera.data.RandomAccessibleIntervalDataSource.<init>(RandomAccessibleIntervalDataSource.java:78)
    at org.janelia.saalfeldlab.paintera.data.n5.N5DataSource.<init>(N5DataSource.java:48)
    at org.janelia.saalfeldlab.paintera.data.n5.N5DataSource.<init>(N5DataSource.java:30)
    at org.janelia.saalfeldlab.paintera.state.label.n5.N5BackendPainteraDataset$Companion.makeSource(N5BackendPainteraDataset.kt:99)
    at org.janelia.saalfeldlab.paintera.state.label.n5.N5BackendPainteraDataset$Companion.access$makeSource(N5BackendPainteraDataset.kt:87)
    at org.janelia.saalfeldlab.paintera.state.label.n5.N5BackendPainteraDataset.createSource(N5BackendPainteraDataset.kt:61)
    at org.janelia.saalfeldlab.paintera.state.label.ConnectomicsLabelState.<init>(ConnectomicsLabelState.kt:98)
    at org.janelia.saalfeldlab.paintera.ui.dialogs.create.CreateDatasetHandler.createAndAddNewLabelDataset(CreateDatasetHandler.java:97)
    at org.janelia.saalfeldlab.paintera.ui.dialogs.create.CreateDatasetHandler.createAndAddNewLabelDataset(CreateDatasetHandler.java:62)
    at org.janelia.saalfeldlab.paintera.ui.dialogs.create.CreateDatasetHandler.createAndAddNewLabelDataset(CreateDatasetHandler.java:46)
    at org.janelia.saalfeldlab.paintera.ui.dialogs.create.CreateDatasetHandler.createAndAddNewLabelDataset(CreateDatasetHandler.java:38)
    at org.janelia.saalfeldlab.paintera.ui.menus.PainteraMenuItems.namedEventHandler$lambda$29$lambda$15(PainteraMenuItems.kt:81)
    at javafx.base@18.0.1/com.sun.javafx.event.CompositeEventHandler.dispatchBubblingEvent(CompositeEventHandler.java:86)
    at javafx.base@18.0.1/com.sun.javafx.event.EventHandlerManager.dispatchBubblingEvent(EventHandlerManager.java:234)
    at javafx.base@18.0.1/com.sun.javafx.event.EventHandlerManager.dispatchBubblingEvent(EventHandlerManager.java:191)
    at javafx.base@18.0.1/com.sun.javafx.event.BasicEventDispatcher.dispatchEvent(BasicEventDispatcher.java:58)
    at javafx.base@18.0.1/com.sun.javafx.event.EventDispatchChainImpl.dispatchEvent(EventDispatchChainImpl.java:114)
    at javafx.base@18.0.1/com.sun.javafx.event.EventUtil.fireEventImpl(EventUtil.java:74)
    at javafx.base@18.0.1/com.sun.javafx.event.EventUtil.fireEvent(EventUtil.java:49)
    at javafx.base@18.0.1/javafx.event.Event.fireEvent(Event.java:198)
    at javafx.controls@18.0.1/javafx.scene.control.MenuItem.fire(MenuItem.java:459)
    at javafx.controls@18.0.1/com.sun.javafx.scene.control.ContextMenuContent$MenuItemContainer.doSelect(ContextMenuContent.java:1385)
    at javafx.controls@18.0.1/com.sun.javafx.scene.control.ContextMenuContent$MenuItemContainer.lambda$createChildren$12(ContextMenuContent.java:1338)
    at javafx.base@18.0.1/com.sun.javafx.event.CompositeEventHandler$NormalEventHandlerRecord.handleBubblingEvent(CompositeEventHandler.java:247)
    at javafx.base@18.0.1/com.sun.javafx.event.CompositeEventHandler.dispatchBubblingEvent(CompositeEventHandler.java:80)
    at javafx.base@18.0.1/com.sun.javafx.event.EventHandlerManager.dispatchBubblingEvent(EventHandlerManager.java:234)
    at javafx.base@18.0.1/com.sun.javafx.event.EventHandlerManager.dispatchBubblingEvent(EventHandlerManager.java:191)
    at javafx.base@18.0.1/com.sun.javafx.event.CompositeEventDispatcher.dispatchBubblingEvent(CompositeEventDispatcher.java:59)
    at javafx.base@18.0.1/com.sun.javafx.event.BasicEventDispatcher.dispatchEvent(BasicEventDispatcher.java:58)
    at javafx.base@18.0.1/com.sun.javafx.event.EventDispatchChainImpl.dispatchEvent(EventDispatchChainImpl.java:114)
    at javafx.base@18.0.1/com.sun.javafx.event.BasicEventDispatcher.dispatchEvent(BasicEventDispatcher.java:56)
    at javafx.base@18.0.1/com.sun.javafx.event.EventDispatchChainImpl.dispatchEvent(EventDispatchChainImpl.java:114)
    at javafx.base@18.0.1/com.sun.javafx.event.BasicEventDispatcher.dispatchEvent(BasicEventDispatcher.java:56)
    at javafx.base@18.0.1/com.sun.javafx.event.EventDispatchChainImpl.dispatchEvent(EventDispatchChainImpl.java:114)
    at javafx.base@18.0.1/com.sun.javafx.event.BasicEventDispatcher.dispatchEvent(BasicEventDispatcher.java:56)
    at javafx.base@18.0.1/com.sun.javafx.event.EventDispatchChainImpl.dispatchEvent(EventDispatchChainImpl.java:114)
    at javafx.base@18.0.1/com.sun.javafx.event.EventUtil.fireEventImpl(EventUtil.java:74)
    at javafx.base@18.0.1/com.sun.javafx.event.EventUtil.fireEvent(EventUtil.java:54)
    at javafx.base@18.0.1/javafx.event.Event.fireEvent(Event.java:198)
    at javafx.graphics@18.0.1/javafx.scene.Scene$MouseHandler.process(Scene.java:3881)
    at javafx.graphics@18.0.1/javafx.scene.Scene.processMouseEvent(Scene.java:1874)
    at javafx.graphics@18.0.1/javafx.scene.Scene$ScenePeerListener.mouseEvent(Scene.java:2607)
    at javafx.graphics@18.0.1/com.sun.javafx.tk.quantum.GlassViewEventHandler$MouseEventNotification.run(GlassViewEventHandler.java:411)
    at javafx.graphics@18.0.1/com.sun.javafx.tk.quantum.GlassViewEventHandler$MouseEventNotification.run(GlassViewEventHandler.java:301)
    at java.base/java.security.AccessController.doPrivileged(Native Method)
    at javafx.graphics@18.0.1/com.sun.javafx.tk.quantum.GlassViewEventHandler.lambda$handleMouseEvent$2(GlassViewEventHandler.java:450)
    at javafx.graphics@18.0.1/com.sun.javafx.tk.quantum.QuantumToolkit.runWithoutRenderLock(QuantumToolkit.java:424)
    at javafx.graphics@18.0.1/com.sun.javafx.tk.quantum.GlassViewEventHandler.handleMouseEvent(GlassViewEventHandler.java:449)
    at javafx.graphics@18.0.1/com.sun.glass.ui.View.handleMouseEvent(View.java:551)
    at javafx.graphics@18.0.1/com.sun.glass.ui.View.notifyMouse(View.java:937)
    at javafx.graphics@18.0.1/com.sun.glass.ui.win.WinApplication._runLoop(Native Method)
    at javafx.graphics@18.0.1/com.sun.glass.ui.win.WinApplication.lambda$runLoop$3(WinApplication.java:184)
    at java.base/java.lang.Thread.run(Thread.java:834)
ddenberg commented 11 months ago

I believe I fixed the issue. I think the block size was too large.

I changed the block size to 64, 64, 16, but I am unsure how this will affect performance. Perhaps someone could recommend a reasonable block size? I am unfamiliar with the data format used to save the labels.

cmhulbert commented 11 months ago

To enable parallel reading/writing, as well as a more responsive UI, the entire label dataset has dimensions as specified, but is chunked based on the specified block size. So in this case, as you initially had, you lose a lot of those benefits if you have the block size the same as the dimensions of the label dataset, since it means in order to read/write, you have to access the entire dataset, as opposed to a specific block. In your case, I think 64x64x16 is reasonable.