Trouble reading LZW compressed TIFF

[ASHISRAVINDRAN]

Hi Tischi, BigDataProcessor is unable to properly read LZW compressed tiff files.

1. I was trying to push our plugin at my lab here, in order to deal with tiff datasets only to find that it fails to open LZW compressed tiffs. Here is the calibrated tiff file which I tried to open using BDP: https://drive.google.com/file/d/1kN_TMJHcyAyQOCegBWVAzJclo2aGZEOv/view?usp=sharing (This tiff opens fine in ImageJ) Initial debugging shows that error is in OpenerExtension.java at https://github.com/tischi/fiji-plugin-bigDataProcessor2/blob/cc0e87c13238adbdbc6e2207a79d48b138e9972e/src/main/java/de/embl/cba/bdp2/OpenerExtension.java#L1066 where the readStart+ readLength value becomes more than the RandomAccessFile length for some z and file data is not read anymore.

2. I also tried to re-save a normal tiff using LZW compression and attemped to load it back. It seems the z slices are not read at all. But there is no error at the above mention line of code.

I am on it but OpenerExtension and related TIFF reading logics are mostly a black box for me. Can you too dig in for a faster fix?

[ASHISRAVINDRAN]

Detailed analysis of the issue(s) revealed 3 hidden issues within the TIFF save code.

• Saving TIFF using LZW compression is flawed since the axis order changed. The newly saved tiff stack has only one frame per stack saved. This is because if the no. of channels is 1, this becomes the new z once the ImagePlus conversion happens.

• Normal, uncompressed, tiff stacks’ saving is implicitly flawed. When we convert the 3D RAI to ImagePlus, again the axis order is switched. But we don’t realize it as the tiff writer anyway saves the new XYC stack (with C=27, for example), instead of XYZ stack. We don’t realize as its cleverly named –C001-T001.tiff. Therefore loading these files next time also don’t cause a problem as the file loading is based on file name format and not on real metadata of the tiff. However, the tiff metadata clearly shows ‘channels = 27’.

• Newly saved Tiff data has no calibration information.

Luckily the fix for all these are the same: call Utils.wrapToCalibratedImagePlus method. :) So, these bugs are fixed now.

Note that the real issue of LZW Tiff (from my lab/working group) loading error is still open. This tiff opens fine in ImageJ(FIJI) directly but not in BDP.