search

ome / bioformats

Bio-Formats is a Java library for reading and writing data in life sciences image file formats. It is developed by the Open Microscopy Environment. Bio-Formats is released under the GNU General Public License (GPL); commercial licenses are available from Glencoe Software.
https://www.openmicroscopy.org/bio-formats
GNU General Public License v2.0
375 stars 241 forks source link

Recent ND2 files cannot be opened by bio-formats with NegativeArraySizeException #3517

Open aarpon opened 4 years ago

aarpon commented 4 years ago

Some recent ND2 files (NIS Elements version 5.2, maybe earlier; you can find an example here: https://drive.google.com/file/d/1nrLmO6KAfUVNf6Ie5YXupZZf5LQLgoBd/view?usp=sharing) fail opening in bio-formats (up to version 6.3.1) with the following exception:

(Fiji Is Just) ImageJ 2.0.0-rc-69/1.52p; Java 1.8.0_172 [64-bit]; Windows 10 10.0; 1798MB of 24263MB (7%)

java.lang.NegativeArraySizeException
    at loci.formats.in.NativeND2Reader.openBytes(NativeND2Reader.java:305)
    at loci.formats.DelegateReader.openBytes(DelegateReader.java:227)
    at loci.formats.ChannelFiller.openBytes(ChannelFiller.java:156)
    at loci.formats.ChannelSeparator.openBytes(ChannelSeparator.java:229)
    at loci.formats.ChannelSeparator.openBytes(ChannelSeparator.java:161)
    at loci.formats.ReaderWrapper.openBytes(ReaderWrapper.java:334)
    at loci.formats.DimensionSwapper.openBytes(DimensionSwapper.java:233)
    at loci.formats.ReaderWrapper.openBytes(ReaderWrapper.java:334)
    at loci.formats.ReaderWrapper.openBytes(ReaderWrapper.java:334)
    at loci.plugins.util.ImageProcessorReader.openProcessors(ImageProcessorReader.java:186)
    at loci.plugins.in.ImagePlusReader.readProcessors(ImagePlusReader.java:422)
    at loci.plugins.in.ImagePlusReader.readPlanes(ImagePlusReader.java:387)
    at loci.plugins.in.ImagePlusReader.readImage(ImagePlusReader.java:282)
    at loci.plugins.in.ImagePlusReader.readImages(ImagePlusReader.java:243)
    at loci.plugins.in.ImagePlusReader.readImages(ImagePlusReader.java:221)
    at loci.plugins.in.ImagePlusReader.openImagePlus(ImagePlusReader.java:116)
    at loci.plugins.in.Importer.readPixels(Importer.java:149)
    at loci.plugins.in.Importer.run(Importer.java:86)
    at loci.plugins.LociImporter.run(LociImporter.java:78)
    at ij.IJ.runUserPlugIn(IJ.java:230)
    at ij.IJ.runPlugIn(IJ.java:193)
    at ij.IJ.runPlugIn(IJ.java:182)
    at HandleExtraFileTypes.openImage(HandleExtraFileTypes.java:499)
    at HandleExtraFileTypes.run(HandleExtraFileTypes.java:72)
    at ij.IJ.runUserPlugIn(IJ.java:230)
    at ij.IJ.runPlugIn(IJ.java:193)
    at ij.IJ.runPlugIn(IJ.java:182)
    at ij.io.Opener.openWithHandleExtraFileTypes(Opener.java:516)
    at ij.io.Opener.openImage(Opener.java:370)
    at ij.io.Opener.openImage(Opener.java:242)
    at ij.io.Opener.open(Opener.java:109)
    at ij.io.Opener.openAndAddToRecent(Opener.java:291)
    at ij.plugin.DragAndDrop.openFile(DragAndDrop.java:192)
    at ij.plugin.DragAndDrop.run(DragAndDrop.java:159)
    at java.lang.Thread.run(Thread.java:748)
dgault commented 4 years ago

Thanks for reporting the bug and providing a sample file. I was able to reproduce the exception using the latest Bio-Formats release (6.3.1) and neither of the ND2 options (chunkmap or native lib) had any impact.

Would it be possible to make the provided sample file available publicly under the CC-BY license?

aarpon commented 4 years ago

Hi,

sorry for the delay: I had to double-check with the owner of the file. The file can be made available publicly under the terms of the CC-BY license. Do I have to do something about it myself?

One more thing: we cropped the image in NIS Elements and the smaller file ( https://drive.google.com/file/d/1_qR5lagexpVq0rgkHvRAFhopBxPCL6ok/view?usp=sharing) could be opened in Fiji successfully.

Thanks, a2

On Fri, Mar 6, 2020 at 2:58 PM David Gault notifications@github.com wrote:

Thanks for reporting the bug and providing a sample file. I was able to reproduce the exception using the latest Bio-Formats release (6.3.1) and neither of the ND2 options (chunkmap or native lib) had any impact.

Would it be possible to make the provided sample file available publicly under the CC-BY license?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/ome/bioformats/issues/3517?email_source=notifications&email_token=AAJLKO43NYQW7O4UMA7RY5LRGD6RTA5CNFSM4LCEI4MKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEOBN6SA#issuecomment-595779400, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAJLKO2YKY7KOJAAL2OAIZLRGD6RTANCNFSM4LCEI4MA .

melissalinkert commented 3 years ago

I believe https://forum.image.sc/t/error-while-opening-nd2-file/44700 is another example of this issue. The problem appears to be specific to files with multiple channels that are stored interleaved when the total pixel count for a plane (X Y channels bits) exceeds 2 GB. Opening smaller tiles is a work-around in the meantime, but we should be able to fix this for the case when X Y * bits is less than 2 GB.

imagesc-bot commented 3 years ago

This issue has been mentioned on Image.sc Forum. There might be relevant details there:

https://forum.image.sc/t/error-while-opening-nd2-file/44700/3

dgault commented 3 years ago

An additional report of this issue comes from forum thread https://forum.image.sc/t/large-nd2-file-doesnt-open-negativearraysizeexception/54897. A new sample file reproduces the exception with Bio-Formats 6.6.1

imagesc-bot commented 3 years ago

This issue has been mentioned on Image.sc Forum. There might be relevant details there:

https://forum.image.sc/t/large-nd2-file-doesnt-open-negativearraysizeexception/54897/5

melissalinkert commented 11 months ago

The file from https://forum.image.sc/t/error-while-opening-nd2-file/44700 no longer has this problem; https://github.com/openmicroscopy/data_repo_config/pull/590 adds it to the test repo.

I was not able to find the other files referenced here in inbox, though, so leaving this open until we can confirm that they are working as well.

dgault commented 4 months ago

A further report of this issue has been received for stitched files on the forum thread: https://forum.image.sc/t/large-nd2-file-doesnt-open-negativearraysizeexception/54897/10