search

bigdataviewer / bigdataviewer-playground

BSD 2-Clause "Simplified" License
19 stars 5 forks source link

Loading CZI file fails #173

Closed haesleinhuepf closed 3 years ago

haesleinhuepf commented 3 years ago

Hi @NicoKiaru and @tischi ,

after your amazing presentation at the Neubias academy last week, I attempted to try the 3D registration tool for slide scanner data. Unfortunately, I can't open CZI files from our slide scanner. Error: "Unhandled 16 bits RGB images". You find the detailed error message below. Do you think this would be easy to fix / implement? I'm happy to do it (that's part of my new job, you know ;-) ), I might just need some guidance and pointers where to start.

Thanks for your support!

Cheers, Robert

When using menu Plugins > BigDataViewer > Playground > BDVDataset > Open [Bioformats bdv bridge (basic)] this error pops up:

Fetcher-0
Fetcher-1
Attempts to set opener settings for file format Zeiss CZI
No name found for serie 16 ch 0
No name found for serie 17 ch 0
Fetcher-0
Fetcher-1
java.lang.UnsupportedOperationException: Unhandled 16 bits RGB images
    at ch.epfl.biop.bdv.bioformats.bioformatssource.BioFormatsBdvSource.getBioformatsBdvSourceType(BioFormatsBdvSource.java:421)
    at ch.epfl.biop.bdv.bioformats.imageloader.BioFormatsImageLoader.lambda$null$3(BioFormatsImageLoader.java:134)
    at java.util.stream.Streams$RangeIntSpliterator.forEachRemaining(Streams.java:110)
    at java.util.stream.IntPipeline$Head.forEach(IntPipeline.java:557)
    at ch.epfl.biop.bdv.bioformats.imageloader.BioFormatsImageLoader.lambda$new$4(BioFormatsImageLoader.java:117)
    at java.util.stream.Streams$RangeIntSpliterator.forEachRemaining(Streams.java:110)
    at java.util.stream.IntPipeline$Head.forEach(IntPipeline.java:557)
    at ch.epfl.biop.bdv.bioformats.imageloader.BioFormatsImageLoader.<init>(BioFormatsImageLoader.java:91)
    at ch.epfl.biop.bdv.bioformats.export.spimdata.BioFormatsConvertFilesToSpimData.getSpimDataInstance(BioFormatsConvertFilesToSpimData.java:283)
    at ch.epfl.biop.bdv.bioformats.export.spimdata.BioFormatsConvertFilesToSpimData.getSpimData(BioFormatsConvertFilesToSpimData.java:312)
    at ch.epfl.biop.bdv.bioformats.command.BasicOpenFilesWithBigdataviewerBioformatsBridgeCommand.run(BasicOpenFilesWithBigdataviewerBioformatsBridgeCommand.java:72)
    at org.scijava.command.CommandModule.run(CommandModule.java:196)
    at org.scijava.module.ModuleRunner.run(ModuleRunner.java:165)
    at org.scijava.module.ModuleRunner.call(ModuleRunner.java:124)
    at org.scijava.module.ModuleRunner.call(ModuleRunner.java:63)
    at org.scijava.thread.DefaultThreadService.lambda$wrap$2(DefaultThreadService.java:225)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
BdvSpimDataPostprocessor:SpimData found.
java.lang.UnsupportedOperationException: Unhandled 16 bits RGB images
    at ch.epfl.biop.bdv.bioformats.bioformatssource.BioFormatsBdvSource.getBioformatsBdvSourceClass(BioFormatsBdvSource.java:440)
    at ch.epfl.biop.bdv.bioformats.bioformatssource.BioFormatsBdvOpener.getConcreteSource(BioFormatsBdvOpener.java:386)
    at ch.epfl.biop.bdv.bioformats.bioformatssource.BioFormatsBdvOpener.getConcreteAndVolatileSources(BioFormatsBdvOpener.java:475)
    at ch.epfl.biop.bdv.bioformats.imageloader.BioFormatsSetupLoader.<init>(BioFormatsSetupLoader.java:91)
    at ch.epfl.biop.bdv.bioformats.imageloader.BioFormatsImageLoader.getSetupImgLoader(BioFormatsImageLoader.java:165)
    at ch.epfl.biop.bdv.bioformats.imageloader.BioFormatsImageLoader.getSetupImgLoader(BioFormatsImageLoader.java:55)
    at sc.fiji.bdvpg.sourceandconverter.SourceAndConverterHelper.createSourceAndConverters(SourceAndConverterHelper.java:186)
    at sc.fiji.bdvpg.scijava.services.SourceAndConverterService.register(SourceAndConverterService.java:212)
    at sc.fiji.bdvpg.scijava.processors.BdvSpimDataPostprocessor.lambda$process$2(BdvSpimDataPostprocessor.java:59)
    at java.util.HashMap.forEach(HashMap.java:1289)
    at sc.fiji.bdvpg.scijava.processors.BdvSpimDataPostprocessor.process(BdvSpimDataPostprocessor.java:54)
    at org.scijava.module.ModuleRunner.postProcess(ModuleRunner.java:114)
    at org.scijava.module.ModuleRunner.run(ModuleRunner.java:174)
    at org.scijava.module.ModuleRunner.call(ModuleRunner.java:124)
    at org.scijava.module.ModuleRunner.call(ModuleRunner.java:63)
    at org.scijava.thread.DefaultThreadService.lambda$wrap$2(DefaultThreadService.java:225)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
[ERROR] Module threw exception
java.lang.NullPointerException
    at ch.epfl.biop.bdv.bioformats.bioformatssource.BioFormatsBdvOpener.getConcreteAndVolatileSources(BioFormatsBdvOpener.java:478)
    at ch.epfl.biop.bdv.bioformats.imageloader.BioFormatsSetupLoader.<init>(BioFormatsSetupLoader.java:91)
    at ch.epfl.biop.bdv.bioformats.imageloader.BioFormatsImageLoader.getSetupImgLoader(BioFormatsImageLoader.java:165)
    at ch.epfl.biop.bdv.bioformats.imageloader.BioFormatsImageLoader.getSetupImgLoader(BioFormatsImageLoader.java:55)
    at sc.fiji.bdvpg.sourceandconverter.SourceAndConverterHelper.createSourceAndConverters(SourceAndConverterHelper.java:186)
    at sc.fiji.bdvpg.scijava.services.SourceAndConverterService.register(SourceAndConverterService.java:212)
    at sc.fiji.bdvpg.scijava.processors.BdvSpimDataPostprocessor.lambda$process$2(BdvSpimDataPostprocessor.java:59)
    at java.util.HashMap.forEach(HashMap.java:1289)
    at sc.fiji.bdvpg.scijava.processors.BdvSpimDataPostprocessor.process(BdvSpimDataPostprocessor.java:54)
    at org.scijava.module.ModuleRunner.postProcess(ModuleRunner.java:114)
    at org.scijava.module.ModuleRunner.run(ModuleRunner.java:174)
    at org.scijava.module.ModuleRunner.call(ModuleRunner.java:124)
    at org.scijava.module.ModuleRunner.call(ModuleRunner.java:63)
    at org.scijava.thread.DefaultThreadService.lambda$wrap$2(DefaultThreadService.java:225)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
NicoKiaru commented 3 years ago

Cool! Awesome that you're back ;-) !

A quick fix is to check 'split RGB channel' on the import command (choose the advanced one). That may work, but then you'll have three independent sources, 16 bits each for red, green and blue channel.

Now all the bioformats bridge is done in the repo https://github.com/BIOP/bigdataviewer-bioformats . And sources 'types' which can be created are defined in this sub package : https://github.com/BIOP/bigdataviewer-bioformats/tree/master/src/main/java/ch/epfl/biop/bdv/bioformats/bioformatssource

As you can see, the supported pixel type are:

It's not too much work to support another type. However, it made sense to use the type ARGBType with 8-bit RGB because it's already heavily used as a native type in imglib2. But there's no type to my knowledge (but maybe I'm wrong) which is ARGB16bitsType in imglib2.

So when you split the RGB component, everything should fall back to 3 UnsignedShortType source. I never tried it but it should work. Let me know how it goes!

haesleinhuepf commented 3 years ago

Hey @NicoKiaru

thanks for the prompt reply! Good hint. Unfortunately, it doesn't work (see error message at the bottom). I'll dive a bit into the code and see if I can fix it. Thanks for the pointers! I'll also try to share the dataset (it's a beautiful one ;-) )

Btw. I love this open-file dialog. It reminds the last file{s} I worked with. That appears like a simple feature but makes my life so much easier :-)

Cheers, Robert,

Fetcher-0
Fetcher-1
ZeissCZIReader initializing C:\structure\temp\slidescanner\2020_01_16__0071.czi
Attempts to set opener settings for file format Zeiss CZI
No name found for serie 16 ch 0
No name found for serie 17 ch 0
Fetcher-0
Fetcher-1
java.lang.UnsupportedOperationException: Unhandled 16 bits RGB images
    at ch.epfl.biop.bdv.bioformats.bioformatssource.BioFormatsBdvSource.getBioformatsBdvSourceType(BioFormatsBdvSource.java:421)
    at ch.epfl.biop.bdv.bioformats.imageloader.BioFormatsImageLoader.lambda$null$3(BioFormatsImageLoader.java:134)
    at java.util.stream.Streams$RangeIntSpliterator.forEachRemaining(Streams.java:110)
    at java.util.stream.IntPipeline$Head.forEach(IntPipeline.java:557)
    at ch.epfl.biop.bdv.bioformats.imageloader.BioFormatsImageLoader.lambda$new$4(BioFormatsImageLoader.java:117)
    at java.util.stream.Streams$RangeIntSpliterator.forEachRemaining(Streams.java:110)
    at java.util.stream.IntPipeline$Head.forEach(IntPipeline.java:557)
    at ch.epfl.biop.bdv.bioformats.imageloader.BioFormatsImageLoader.<init>(BioFormatsImageLoader.java:91)
    at ch.epfl.biop.bdv.bioformats.export.spimdata.BioFormatsConvertFilesToSpimData.getSpimDataInstance(BioFormatsConvertFilesToSpimData.java:283)
    at ch.epfl.biop.bdv.bioformats.export.spimdata.BioFormatsConvertFilesToSpimData.getSpimData(BioFormatsConvertFilesToSpimData.java:312)
    at ch.epfl.biop.bdv.bioformats.command.OpenFilesWithBigdataviewerBioformatsBridgeCommand.run(OpenFilesWithBigdataviewerBioformatsBridgeCommand.java:64)
    at org.scijava.command.CommandModule.run(CommandModule.java:196)
    at org.scijava.module.ModuleRunner.run(ModuleRunner.java:165)
    at org.scijava.module.ModuleRunner.call(ModuleRunner.java:124)
    at org.scijava.module.ModuleRunner.call(ModuleRunner.java:63)
    at org.scijava.thread.DefaultThreadService.lambda$wrap$2(DefaultThreadService.java:225)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
BdvSpimDataPostprocessor:SpimData found.
ZeissCZIReader initializing C:\structure\temp\slidescanner\2020_01_16__0071.czi
[ERROR] temp file rename returned false: C:\structure\temp\slidescanner\.2020_01_16__0071.czi.bfmemo1453919627800904570
ZeissCZIReader initializing C:\structure\temp\slidescanner\2020_01_16__0071.czi
[ERROR] temp file rename returned false: C:\structure\temp\slidescanner\.2020_01_16__0071.czi.bfmemo4692538880164564976
ZeissCZIReader initializing C:\structure\temp\slidescanner\2020_01_16__0071.czi
[ERROR] temp file rename returned false: C:\structure\temp\slidescanner\.2020_01_16__0071.czi.bfmemo3014653378616544185
ZeissCZIReader initializing C:\structure\temp\slidescanner\2020_01_16__0071.czi
[ERROR] temp file rename returned false: C:\structure\temp\slidescanner\.2020_01_16__0071.czi.bfmemo1792395738247481865
ZeissCZIReader initializing C:\structure\temp\slidescanner\2020_01_16__0071.czi
[ERROR] temp file rename returned false: C:\structure\temp\slidescanner\.2020_01_16__0071.czi.bfmemo7073722146548157260
ZeissCZIReader initializing C:\structure\temp\slidescanner\2020_01_16__0071.czi
[ERROR] temp file rename returned false: C:\structure\temp\slidescanner\.2020_01_16__0071.czi.bfmemo5730550193454076866
ZeissCZIReader initializing C:\structure\temp\slidescanner\2020_01_16__0071.czi
[ERROR] temp file rename returned false: C:\structure\temp\slidescanner\.2020_01_16__0071.czi.bfmemo1279685714103751666
ZeissCZIReader initializing C:\structure\temp\slidescanner\2020_01_16__0071.czi
[ERROR] temp file rename returned false: C:\structure\temp\slidescanner\.2020_01_16__0071.czi.bfmemo1613551509278323964
ZeissCZIReader initializing C:\structure\temp\slidescanner\2020_01_16__0071.czi
[ERROR] temp file rename returned false: C:\structure\temp\slidescanner\.2020_01_16__0071.czi.bfmemo6438908773133443787
ZeissCZIReader initializing C:\structure\temp\slidescanner\2020_01_16__0071.czi
[ERROR] temp file rename returned false: C:\structure\temp\slidescanner\.2020_01_16__0071.czi.bfmemo6556887228661804771
[ERROR] Module threw exception
java.lang.NullPointerException
    at sc.fiji.bdvpg.sourceandconverter.SourceAndConverterHelper.createSourceAndConverters(SourceAndConverterHelper.java:244)
    at sc.fiji.bdvpg.scijava.services.SourceAndConverterService.register(SourceAndConverterService.java:212)
    at sc.fiji.bdvpg.scijava.processors.BdvSpimDataPostprocessor.lambda$process$2(BdvSpimDataPostprocessor.java:59)
    at java.util.HashMap.forEach(HashMap.java:1289)
    at sc.fiji.bdvpg.scijava.processors.BdvSpimDataPostprocessor.process(BdvSpimDataPostprocessor.java:54)
    at org.scijava.module.ModuleRunner.postProcess(ModuleRunner.java:114)
    at org.scijava.module.ModuleRunner.run(ModuleRunner.java:174)
    at org.scijava.module.ModuleRunner.call(ModuleRunner.java:124)
    at org.scijava.module.ModuleRunner.call(ModuleRunner.java:63)
    at org.scijava.thread.DefaultThreadService.lambda$wrap$2(DefaultThreadService.java:225)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)

and

(Fiji Is Just) ImageJ 2.1.0/1.53c; Java 1.8.0_172 [64-bit]; Windows 10 10.0; 209MB of 18147MB (1%)

java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.NullPointerException
    at net.imagej.legacy.LegacyService.runLegacyCompatibleCommand(LegacyService.java:307)
    at net.imagej.legacy.DefaultLegacyHooks.interceptRunPlugIn(DefaultLegacyHooks.java:166)
    at ij.IJ.runPlugIn(IJ.java)
    at ij.Executer.runCommand(Executer.java:150)
    at ij.Executer.run(Executer.java:68)
    at java.lang.Thread.run(Thread.java:748)
Caused by: java.util.concurrent.ExecutionException: java.lang.NullPointerException
    at java.util.concurrent.FutureTask.report(FutureTask.java:122)
    at java.util.concurrent.FutureTask.get(FutureTask.java:192)
    at net.imagej.legacy.LegacyService.runLegacyCompatibleCommand(LegacyService.java:303)
    ... 5 more
Caused by: java.lang.NullPointerException
    at sc.fiji.bdvpg.sourceandconverter.SourceAndConverterHelper.createSourceAndConverters(SourceAndConverterHelper.java:244)
    at sc.fiji.bdvpg.scijava.services.SourceAndConverterService.register(SourceAndConverterService.java:212)
    at sc.fiji.bdvpg.scijava.processors.BdvSpimDataPostprocessor.lambda$process$2(BdvSpimDataPostprocessor.java:59)
    at java.util.HashMap.forEach(HashMap.java:1289)
    at sc.fiji.bdvpg.scijava.processors.BdvSpimDataPostprocessor.process(BdvSpimDataPostprocessor.java:54)
    at org.scijava.module.ModuleRunner.postProcess(ModuleRunner.java:114)
    at org.scijava.module.ModuleRunner.run(ModuleRunner.java:174)
    at org.scijava.module.ModuleRunner.call(ModuleRunner.java:124)
    at org.scijava.module.ModuleRunner.call(ModuleRunner.java:63)
    at org.scijava.thread.DefaultThreadService.lambda$wrap$2(DefaultThreadService.java:225)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    ... 1 more
NicoKiaru commented 3 years ago

Too bad it doesn't work! If you can share it, I can have a look also.

Btw. I love this open-file dialog. It reminds the last file{s} I worked with.

Pure File[] scijava parameter ;-)

imagejan commented 3 years ago

Pure File[] scijava parameter ;-)

Glad you like it! 😉

NicoKiaru commented 3 years ago

@haesleinhuepf I fixed the issue in https://github.com/BIOP/bigdataviewer-bioformats/commit/8aa4ca2d13696b73949a88260bfc017f7d90687e and uploaded to the update site. You can try again and reopen the issue if it doesn't work.

You'll need to split the RGB channels in the command, but it should work this time (and I've added the checkbox in the basic command). A little issue is that the three channels will be initialized to white - you'll have to manually set the channels red, green and blue (then you can resave the xml dataset file, if you want to avoid doing this all the time). I've put an issue about this 'wrong colors' : https://github.com/BIOP/bigdataviewer-bioformats/issues/37