Grid/Collection Stitching Plugin Gives Array Index Out of Bounds Error

[iomega15]

hello, we are trying to stitch two 3d tiff stacks (“tiles”): https://www.dropbox.com/sh/p26swfkmtkw772m/AACgTqMCWAC5gUDF8PPwhMKpa?dl=0

(the tile 140 should be above tile 1)

but no matter what inputs we try, we keep getting an out of bounds error when using a position file the grid stitcher.

we tried two tiles, four, and 140 tiles. we tried relaxed and stringent displacement thresholds, we tried different signs of the tile offsets in the position file.

nothing helps. also the PC has 64GB of ram, and when we look at the task manager, its not a resources issue.

finally, the pairwise stitcher is able to stitch these two stacks perfectly (see pairwise_fused.tiff). so it appears that there is some error in the grid stitcher. please help. thanks!

[StephanPreibisch]

Hi, the reason is that you assign 3d locations in the tileconfiguration to 2d+t images that you process, which fails. You can either modify the metadata of your input (Image > Properties) to have slices instead of timepoints, or work in 2d-mode and provide 2d locations. From the image data it is not clear to me if it is 2d+t or "real" 3d data (x,y,z). Hope this helps?

[iomega15]

Hi Stephan,

Thanks for your guidance! We are interested in 3D mode. I thought it was strange that the code treats 3D tiff stack "pages" as time steps. Can you tell us what exactly we need to change in the metadata in order to interpret the data as slices instead? In any case, the code should give a more meaningfull error than out of bounds. Thanks

On Mon, Sep 4, 2017 at 14:06 Stephan Preibisch notifications@github.com wrote:

Hi, the reason is that you assign 3d locations in the tileconfiguration to 2d+t images that you process, which fails. You can either modify the metadata of your input (Image > Properties) to have slices instead of timepoints, or work in 2d-mode and provide 2d locations. From the image data it is not clear to me if it is 2d+t or "real" 3d data (x,y,z). Hope this helps?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/fiji/Stitching/issues/47#issuecomment-327009975, or mute the thread https://github.com/notifications/unsubscribe-auth/AFBVO-u_oClgw2mXZ17BWFvIejH8EV_rks5sfDwdgaJpZM4PLONz .

-- null

[iomega15]

Hello, we have an update. We could not find the syntax of the metadata that IJ expects documented anywhere, so we figured it out by comparing the metadata of the pair-wise stitching plugin's output (which surprisingly resulted slices, even though it stitched 2d+t images). In the process, we discovered another bug: apparently, if windows explorer is used to edit the tiff metadata, it compresses 32 bit images to 16 bits. Although IJ would refuse to open the compressed tiffs, the grid-stitcher's "load tiles from a positions file" feature loads such tiffs without a warning, and ultimately leads to the attached Null Pointer exception.

On Mon, Sep 4, 2017 at 2:31 PM, Roman Voronov bopohob@gmail.com wrote:

Hi Stephan,

Thanks for your guidance! We are interested in 3D mode. I thought it was strange that the code treats 3D tiff stack "pages" as time steps. Can you tell us what exactly we need to change in the metadata in order to interpret the data as slices instead? In any case, the code should give a more meaningfull error than out of bounds. Thanks

On Mon, Sep 4, 2017 at 14:06 Stephan Preibisch notifications@github.com wrote:

Hi, the reason is that you assign 3d locations in the tileconfiguration to 2d+t images that you process, which fails. You can either modify the metadata of your input (Image > Properties) to have slices instead of timepoints, or work in 2d-mode and provide 2d locations. From the image data it is not clear to me if it is 2d+t or "real" 3d data (x,y,z). Hope this helps?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/fiji/Stitching/issues/47#issuecomment-327009975, or mute the thread https://github.com/notifications/unsubscribe-auth/AFBVO-u_oClgw2mXZ17BWFvIejH8EV_rks5sfDwdgaJpZM4PLONz .

-- null

(Fiji Is Just) ImageJ 2.0.0-rc-61/1.51n; Java 1.8.0_66 [64-bit]; Windows 8.1 6.3; 414MB of 49030MB (<1%)

java.lang.NullPointerException

at mpicbg.imglib.algorithm.fft.PhaseCorrelationPeak.getOriginalInvPCMPosition(PhaseCorrelationPeak.java:74)

at mpicbg.imglib.algorithm.fft.PhaseCorrelation.verifyWithCrossCorrelation(PhaseCorrelation.java:261)

at mpicbg.imglib.algorithm.fft.PhaseCorrelation.process(PhaseCorrelation.java:224)

at mpicbg.stitching.PairWiseStitchingImgLib.computePhaseCorrelation(PairWiseStitchingImgLib.java:231)

at mpicbg.stitching.PairWiseStitchingImgLib.performStitching(PairWiseStitchingImgLib.java:217)

at mpicbg.stitching.PairWiseStitchingImgLib.stitchPairwise(PairWiseStitchingImgLib.java:138)

at mpicbg.stitching.CollectionStitchingImgLib$1.run(CollectionStitchingImgLib.java:90)

at java.lang.Thread.run(Thread.java:745)

at mpicbg.imglib.multithreading.SimpleMultiThreading.startAndJoin(SimpleMultiThreading.java:139)

at mpicbg.stitching.CollectionStitchingImgLib.stitchCollection(CollectionStitchingImgLib.java:112)

at plugin.Stitching_Grid.run(Stitching_Grid.java:539)

at ij.IJ.runUserPlugIn(IJ.java:217)

at ij.IJ.runPlugIn(IJ.java:181)

at ij.Executer.runCommand(Executer.java:137)

at ij.Executer.run(Executer.java:66)

at java.lang.Thread.run(Thread.java:745)

[StephanPreibisch]

Hi, I checked out your original TIFFs now as well and there is something off with the format. How did you produce these files, is it 16-bit float? Using Bioformats I can open it, but it does look funny and it pretends that it is 2d over time, not 3d (see screenshot).

[StephanPreibisch]

So the first step is to import your TIFFs correctly, fix the metadata (147 slices, not 147 timepoints), then everything else will just work I think ... You can open this window by choosing (Image > Properties). Using ImageJ Macro Scripting (Plugins > Macro > Record) you can easily automate this process for all images.

[axtimwalde]

Hi @StephanPreibisch . I assume this is the long standing inconsistency in how ImageJ and Bioformats handle TIFF files without meta-data. ImageJ's opener shows them as 3D xyz, Bioformats defaults to 2D+t which is ... one of those things. I have hack-fixed this for people here but I am not sure if this wouldn't break other stuff. Generally, it would be nice if Bioformats wouldn't do this. Here is the hack, would be great if you could have a look and decide if this is a good or bad idea:

https://github.com/fiji/Stitching/commit/07b8c02289fbb34faa5a2031722cdbf9c22c55c8

line 103 and following use the ImageJ opener which does not have this issue. Line 113 and following hard force 2D time series to be 3D volumes (may be not a good idea).

[StephanPreibisch]

Hi @axtimwalde, thanks! But for me ImageJ itself doesn't open it at all saying "cannot open 16-bit float". So I am not sure if there is a second problem on top of what you described now ...

[imagejan]

@axtimwalde did you test your fix with non-tiff images (e.g. czi, nd, stk) as well? In my experience, the line:

final ImagePlus impIj = new Opener().openImage( file.getAbsolutePath() );

will redirect to Bio-Formats if IJ1's own methods fail, and then show the Bio-Formats import dialog 😄 You could maybe use new Opener().openTiff() instead...

I agree with you that it would be best if bio-formats wouldn't handle those differently than IJ1 alone. But I think BF does some special casing for IJ-written tiffs with metadata at least. Where are the metadata-less tiffs coming from?

[iomega15]

@StephanPreibisch , the "cannot open 16-bit float" is the second problem that I described in my "update":

• windows explorer changes the 32-bit tiff to 16 bit, when we try to use it for editing the tiff metadata -the grid-stitcher tries to open these files when supplied through the position file, even though IJ would not open them -Null Point exception occurs in the Grid Stitcher

I guess I should have opened a different ticket for this problem. Also, I am sorry that my original dropbox link contained the tainted 16bit tiff. Here are the truly-original 32-bit tiffs, that have NOT been altered by windows explorer:

https://www.dropbox.com/s/bzv1vdfnpovybt9/FIBERS_tile_00001.tif?dl=0 https://www.dropbox.com/s/t1svymxw1juq8oc/FIBERS_tile_00140.tif?dl=0

IJ does load these one without errors, but still considers them to be 2D+t images (because they lack metadata), instead of 3D.

[StephanPreibisch]

With those files I understand what @axtimwalde meant. Well, this is a problem and is caused by an inconsistent behaviour between Bioformats and the ImageJ Opener as Saalfeld pointed out - maybe there are good reasons for that. Anyhow, I am inclined to merge the hack-fix by @axtimwalde. Any objections @ctrueden? Or is there any chance to fix that on the side of Bioformats? Thanks a lot, Stephan

[ctrueden]

I could have sworn I already added a hack in the Stitching code to handle this situation, but it has been a long time.

The reason that Bio-Formats treats plain 3D TIFFs as XYT is that back in the day, LOCI's plain TIFFs were indeed XYT, not XYZ.

The solution to this inconsistency in the future will be to support explicit "unspecified" or "unknown" dimensions which can be handled differently—SCIFIO supports such dimensions directly. Bio-Formats also does already support this idea indirectly via IFormatReader#isOrderCertain() which gives you a big hint that that T dimension is not necessarily actually T. In the short term, I think it would make most sense to call isOrderCertain() and ask the user in that case. Otherwise you will just break other flavors of TIFF file. (In general, this is why plugins like Stitching should not even be invoking Bio-Formats directly, but relying on data which is already open. But I understand that in this case, the IJ1 paradigm does not support this.)

[StephanPreibisch]

Hi Curtis, since it is potentially many files that do not fit into memory at once, having all open at the same time is not really possible. I will check out calling IFormatReader#isOrderCertain() ... this sounds like the best way to go. Thanks a lot!

[ctrueden]

since it is potentially many files that do not fit into memory at once, having all open at the same time is not really possible.

I agree. My point is that we need to change the paradigm of ImageJ. Having a file "open" should not mean having all samples in memory. And having a mosaic "open" should not mean having one image window per tile.

Sounds good about isOrderCertain(); let me know if you have any questions about it.

[StephanPreibisch]

Hopefully fixed using commit https://github.com/fiji/Stitching/commit/cc96f5fdd6b8fefefe31710c6801dcd9213964b0

Already uploaded it to Fiji as well.

It did not stitch it right though, but that's maybe a different story.

[StephanPreibisch]

Oh yeah, absolutely, that would be great @ctrueden ... basically how it is when using the SpimData object (specifically with HDF5-like datasets) underlying the BDV.

[lacan]

Hi sorry to reopen this thread, but we are having an issue with this workaround... We have XYT series that are saved as such using the standard Fiji Save As Tiff command.

Now we'd like to stitch them but it misinterprets the order, and stitching fails...

So how can we make sure that isOrderCertain returns True for a TIFF saved with Fiji?

[ctrueden]

@lacan I can reproduce that issue:

• Open Mitosis sample
• Save the Mitosis sample to TIFF
• From the CLI, call showinf -nopix mitosis.tif

Bio-Formats says: "Dimension order = XYCZT (uncertain)" which is not correct in this case, since the multi-dimensional ImageJ 1.x metadata is present (and can be seen via tiffcomment mitosis.tif), and therefore the order should be certain.

Report this as an issue to the Bio-Formats team?

[NicoKiaru]

I can confirm that this issue is still present.