jeanollion / bacmman

GNU General Public License v3.0
11 stars 0 forks source link

Duplicate Key error #3

Closed j-broughton closed 3 years ago

j-broughton commented 4 years ago

Hi,

I'll start by saying that this software is great and a pleasure to use compared to others I have tried.

When loading some datasets I get the error below, and I am having a hard time troubleshooting this. I have no experience with java by the way. Looking this up online this error appears to have been fixed in later versions of java, but I am not sure how to implement different versions with Fiji.

I suspect that there are too many files being imported? Because I can separate the data set into different folders and load those just fine.

Exception in thread "AWT-EventQueue-0" java.lang.IllegalStateException: Duplicate key C:\MATLAB\20191222_MMC_Gly_206_302\20191222_MMC_Gly_206_302_0002_TIF_302\IMG-20191222_MMC_Gly_206_302_XY-XY20_302_CHN-mCherry_IT-0001.tif at java.util.stream.Collectors.lambda$throwingMerger$0(Collectors.java:133) at java.util.HashMap.merge(HashMap.java:1254) at java.util.stream.Collectors.lambda$toMap$58(Collectors.java:1320) at java.util.stream.ReduceOps$3ReducingSink.accept(ReduceOps.java:169) at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1382) at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481) at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471) at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708) at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499) at bacmman.data_structure.image_container.MultipleImageContainerPositionChannelFrame.lambda$createFileMap$5(MultipleImageContainerPositionChannelFrame.java:267) at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184) at java.util.stream.SortedOps$SizedRefSortingSink.end(SortedOps.java:352) at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482) at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471) at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151) at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174) at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418) at bacmman.data_structure.image_container.MultipleImageContainerPositionChannelFrame.createFileMap(MultipleImageContainerPositionChannelFrame.java:266) at bacmman.data_structure.image_container.MultipleImageContainerPositionChannelFrame.(MultipleImageContainerPositionChannelFrame.java:112) at bacmman.data_structure.image_container.MultipleImageContainerPositionChannelFrame.(MultipleImageContainerPositionChannelFrame.java:142) at bacmman.core.ImageFieldFactory.importImagesCTP(ImageFieldFactory.java:260) at bacmman.core.ImageFieldFactory.importImages(ImageFieldFactory.java:73) at bacmman.data_structure.Processor.importFiles(Processor.java:72) at bacmman.ui.GUI.importImagesMenuItemActionPerformed(GUI.java:3375) at bacmman.ui.GUI.access$6800(GUI.java:128) at bacmman.ui.GUI$86.actionPerformed(GUI.java:2318) at javax.swing.AbstractButton.fireActionPerformed(AbstractButton.java:2022) at javax.swing.AbstractButton$Handler.actionPerformed(AbstractButton.java:2348) at javax.swing.DefaultButtonModel.fireActionPerformed(DefaultButtonModel.java:402) at javax.swing.DefaultButtonModel.setPressed(DefaultButtonModel.java:259) at javax.swing.AbstractButton.doClick(AbstractButton.java:376) at javax.swing.plaf.basic.BasicMenuItemUI.doClick(BasicMenuItemUI.java:842) at javax.swing.plaf.basic.BasicMenuItemUI$Handler.mouseReleased(BasicMenuItemUI.java:886) at java.awt.Component.processMouseEvent(Component.java:6539) at javax.swing.JComponent.processMouseEvent(JComponent.java:3324) at java.awt.Component.processEvent(Component.java:6304) at java.awt.Container.processEvent(Container.java:2239) at java.awt.Component.dispatchEventImpl(Component.java:4889) at java.awt.Container.dispatchEventImpl(Container.java:2297) at java.awt.Component.dispatchEvent(Component.java:4711) at java.awt.LightweightDispatcher.retargetMouseEvent(Container.java:4904) at java.awt.LightweightDispatcher.processMouseEvent(Container.java:4535) at java.awt.LightweightDispatcher.dispatchEvent(Container.java:4476) at java.awt.Container.dispatchEventImpl(Container.java:2283) at java.awt.Window.dispatchEventImpl(Window.java:2746) at java.awt.Component.dispatchEvent(Component.java:4711) at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:760) at java.awt.EventQueue.access$500(EventQueue.java:97) at java.awt.EventQueue$3.run(EventQueue.java:709) at java.awt.EventQueue$3.run(EventQueue.java:703) at java.security.AccessController.doPrivileged(Native Method) at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:74) at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:84) at java.awt.EventQueue$4.run(EventQueue.java:733) at java.awt.EventQueue$4.run(EventQueue.java:731) at java.security.AccessController.doPrivileged(Native Method) at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:74) at java.awt.EventQueue.dispatchEvent(EventQueue.java:730) at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:205) at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:116) at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:105) at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101) at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:93) at java.awt.EventDispatchThread.run(EventDispatchThread.java:82)

Also, will the recent changes you have been making be added when I update through fiji?

Thanks in advance.

jeanollion commented 4 years ago

Hello @JBroughton80 , Thanks for your encouragements. With this import mode, bacmman split all you files by position name then channel index then frame.

So it seems there is at least 2 files that have the same detected position name & channel index & frame. I will try to take the time to write a more human-readable error message !!

Position name / channel index / frame are identified according to the keywords you configure (cf hint of the import method), so check the import settings and make sure they are set so that position name / channel index and frame are uniquely and well identified. For instance for this file : IMG-20191222_MMC_Gly_206_302_XY-XY20_302_CHN-mCherry_IT-0001.tif , supposing the position number is the 20 located after the XY do not use XY as position identifier as it is not unique in the name but XY-XY or even maybe _IMG-20191222_MMC_Gly_206_302XY-XY if it is shared by all the files of your dataset and for instance you can use _IT- as frame identifier. I'll update the hint of the method to make it clearer.

Also check your dataset for duplicate files (in terms of position name / channel index and frame) , start with the file IMG-20191222_MMC_Gly_206_302_XY-XY20_302_CHN-mCherry_IT-0001.tif

Some bug fixes have been made but nothing to do with image import, I will update them in FIJI shortly. Good luck, Jean

j-broughton commented 4 years ago

Thanks for your quick response.

I have checked that there are no duplicates.

I managed to import everything by separating the data into separate folders, and then importing each folder.

Thanks, James