Error during import of TIFF files

[liorgolgher]

Greetings,

Thank you for offering FLIMfit!

I have tried to import a TIFF file to FLIMfit 5.0.3 for the first time. It's a 512x512x10 matrix, obviously with 10 time channels, each 800 ps wide. I have received the following error message:

---

Error using horzcat
Dimensions of matrices being concatenated are not consistent.

Error in flim_data_series/get_image_dimensions (line 289)

Error in flim_data_series/load_multiple (line 30)

Error in flim_data_series/load_single (line 104)

Error in flim_data_series_controller/load_single (line 167)

Error in front_end_menu_controller/menu_file_load_single_callback (line 654)

Error in front_end_menu_controller>@(varargin)obj.menu_file_load_single_callback(varargin{:})

Error in escaped_callback (line 7)

Error in front_end_menu_controller>@(x,y)escaped_callback(fcn) (line 261)

---

Please let me know how should I resolve it.

Thanks, Lior

[seanwarren]

Hi Lior,

How are you creating your tiff files? Please could you send me a copy of one of the files that you're having problems loading at s.warren@garvan.org.au.

Cheers Sean

[seanwarren]

Hi Lior,

Thanks for sending the file. FLIMfit can't open 'raw' tif files as it needs more information, e.g. what the timepoints of each image are. There are a few different options depending on how you are creating your data. If you are working in Matlab I have a script that can create a 'ffh' file that FLIMfit can read easily, or alternatively you can save an OME tif which can contain the timepoint data. Finally, you can save a series of single plane tif files with filenames that end with the timepoint of each gate, e,g T01000 for 1000ps. If you let me know how you are creating the file we can work out the easiest way to import your data.

Cheers Sean

[liorgolgher]

Hi Sean,

We've migrated from Matlab to Python [1], but could probably extend our program to generate ffh files. Could you please share with us an exemplary ffh file, and possibly your Matlab script that generates it?

Our Python program actually outputs six-dimensional HDF5 files, which I later convert to TIFF using FIJI in order to load into FLIMfit. I was about to ask if FLIMfit will support HDF5, so I've first browsed through your issue folder and came across a similar request raised a month ago [2] (: Indeed it might make sense to use a common data structure for Photon-HDF5, PySight and FLIMfit, at least until ImageJ2 matures as a common platform for all three toolkits [3].

Thanks a lot! Lior

[1] https://pypi.python.org/pypi/pysight

[2] https://github.com/flimfit/FLIMfit/issues/334

[3] https://arxiv.org/abs/1701.05940

[imunro]

Hi Lior One possibility is to use ome-tiff. We write our data as ome-tiff using the bio-formats library and there are details of how to use this from Python at https://docs.openmicroscopy.org/bio-formats/5.7.1/developers/python-dev.html.

Ian

[dgault]

Hi @liorgolgher,

I am a developer on the Bio-Formats team. As suggested OME-TIFF could be one possibility for you. If you wished to write OME-TIFF directly using the Python wrapper I would be happy to assist with any questions you may have.

Also you mentioned that you were currently converting to TIFF via FIJI. I have not tested it using the same workflow as yourself but it may be possible to also export to OME-TIFF using Plugins -> Bio-Formats -> Bio-Formats Exporter

David

[imunro]

Thanks for your thoughts @dgault.

My understanding is that the Fiji->ome.tiff workflow that you suggest will fail for the same reasons that the Fiji->.tif one did ie FLIMfit needs a little more information. Looking at the openmicroscopy documentation it should be possible to add this by editing the .xml. See https://docs.openmicroscopy.org/bio-formats/5.7.1/users/comlinetools/edit.html The missing .xml is described at https://docs.openmicroscopy.org/ome-model/5.5.7/developers/6d-7d-and-8d-storage.html

The workflow would then be: 1) Export to ome-tiff as a time-series using Plugins -> Bio-Formats -> Bio-Formats Exporter 2) Add missing .xml -something like ModuloAlongT Type="lifetime" TypeDescription="TCSPC" Start="0" Step="2" End="128"/ with appropriate formatting as above.

This might be worth a try to allow you to evaluate FLIMfit with your data. I think this would all be scriptable if needed.

[imunro]

@liorgolgher I attach a screen shot of my attempt at the missing xml for your data (the text doesn't seem to display in here I'm afraid)

[seanwarren]

Hi all,

I agree OME-TIFF is a good way forward in principle, however I'm not sure it's trivial to achieve in practice. I think it would be good if we could come up with a couple of helper functions in Matlab/Python/etc to easily export a data cube to a valid ModuloAlongT OME-TIFF that FLIMfit would read. @imunro - do you have any matlab code to export to OME-TIFF at the moment?

@liorgolgher - I'm happy to look at supporting HDF5 if there is an agreed format. As discussed in #334, the photon-hdf5 format does not currently support image-based time resolved data. How do you store your HDF5 data?

[imunro]

Hi Sean

I think exporting from Fiji and then editing as described, while possibly not a good long-term solution, should be very easy to do as a short-term fix. Yes I do have both Matlab and (I think) Java code to write FLIM ome.tiff. Happy to make those available

[seanwarren]

Great - I get asked every now and then what the easiest way to get data from Matlab/Python etc into FLIMfit is and I haven't really had a good answer. It would be good to have some examples online to point to.

[imunro]

We could, of course, prompt the user, when the timing information is missing, to enter it manually. I've been hesitant to do that as it's not a good solution generally.

[HagaiHargil]

Hi everyone,

I'm the author of PySight, which parses .lst files generated by FAST ComTec's multiscaler. List files contain a timestamp for each event with sub-nanosecond precision. PySight essentially places photon-events in their respective bin of a multidimensional histogram - a 5 (6) dimensional array (x-y-z-lifetime-time(-spectral channel)). The "bins", or edges, of the histograms are calculated from events generated by the scanning modules, also recorded in the list file. Our main target audience as of now is the neuroimaging community.

In regards to our data format, after some deliberations we settled on a simple compression of the multidimensional array to a .hdf5 file. One of the "attributes" saved in the file lists the order of recorded dimensions. I believe @tritemio mentions a slightly different implementation here, but the differences between us are minimal. It will be fantastic if we could settle on one format for multidimensional FLIM analysis, as I believe it's a common interest for us :)

That being said - our current format is far from perfect. I would very much like to use a Fiji-based plugin to visualize the multidimensional data, but as @tritemio might be aware, viewing .hdf5 files in Fiji is currently very limited. The only existing plugin isn't maintained and doesn't parse metadata. Other visualization solutions, like the HDF5Viewer, don't come anywhere near Fiji in terms of data analysis and widespread adoption. I believe Photon-HDF5 recommends this Viewer, but I guess it's just due to the lack of better options.

To summarize this much-too-long comment, I believe we have two tasks at hand:

1. First, we'd like to "feel the power" of FLIMfit by manually manipulating of one of our datasets into a suitable format for it to read. To this end, either me or @liorgolgher would be happy to receive @imunro's script and forcefully feed FLIMfit with our images. From the available demonstrations it seems that FLIMfit is more than enough for our needs :)

2. After everything Just Works we stand at a crossroads. If FLIMfit can also help with visualization - it should be our go-to tool, and I think the Photon-HDF5 group should hop right in as well. Else, we diverge - FLIMfit will remain our tool to analyze the FLIM dimension of the multidimensional histogram (and will be granted with full credit in our upcoming paper), but the other dimensions will require a better visualization tool - a problem we'll have to resolve with the Photon-HDF5 guys :)

Looking forward to hear your comments and get this collaboration underway.

[seanwarren]

Hi @HagaiHargil, thanks for your input, it's good to get a clearer idea of what PySight is doing.

Firstly, I agree that the simplest thing will be to write OME-TIFFs using a script. I'm going to try and put together a couple of example scripts in the next few days. It looks like the bioformats-python package might need a little bit of modification to support writing FLIM data as I don't think you can currently add custom structured annotations (@dgault - unless I have missed something?).

In the longer term, I am currently finishing up some reasonably significant modifications of FLIMfit to allow multispectral FLIM analysis. Depending on what you're trying to achieve in terms of multidimensional analysis this might be useful for you and I'm happy to discuss this further.

Finally, just to let you know I maintain a generalised c++ library for reading in FLIM data recorded in a time-tagged mode like the multiscaler you describe at https://github.com/flimfit/FlimReader. Currently it supports Picoquant, B&H and cronologic FLIM systems. It reassembles the images using markers from the microscope scanner, presumably in a similar manner to PySight. We use this to read in time-tagged data directly into FLIMfit. It may be possible to support data from the multiscalar directly using this library. Since you already have PySight this probably wouldn't be useful but I thought I'd mention it anyway as there may be features that could be usefully transplanted in one direction or the other.

Cheers Sean

[imunro]

Hi @seanwarren

As requested an example of writing a FLIM ome-tiff from a Matlab array can be found at https://github.com/imunro/ic-converterM/blob/master/testOMETIFFGenerate.m. The central portion of the code (clearly marked) can be omitted if you have data in the array already as it simply puts some human-readable data into the array for test purrposes. The example generated is a Z-stack with 3 time-points at each pixel.

In order to use you will need to download the bio-formats Matlab plugin from http://www.openmicroscopy.org/bio-formats/downloads/ and amend the paths accordingly. Please note that this code was intended for in-house testing and is not of a "production" standard. There are a number of other examples in the same repo if needed.

[imunro]

Hi Again

An example of writing a FLIM ome-tiff from Java can be found at https://github.com/imunro/FileWriteSPW/blob/master/SPW/FileWrite.java Please note: this is fairly old code and will almost certainly need updating to work with new versions of bio-formats.

For completeness: there is an example of writing a FLIM multiwell-plate data to an ome-tiff at https://github.com/openmicroscopy/bioformats/blob/develop/components/formats-gpl/utils/FileWriteSPW.java

[imunro]

@liorgolgher I've just been trying to use Fiji and the bio-formats command line tools to generate a FLIM ome.tif, as described above. Do you think you could send me a copy of one of your data files to try this with, along with a description of any extra steps needed to open it in Fiji Many thanks.

[liorgolgher]

Thank you Ian! The files were too large to email so I've shared them via Google Drive. Please let me know if any further details or files could be helpful.

Thanks, Lior

[imunro]

Thanks Lior. Got them I'll let you know how it goes.

[imunro]

Hi Lior I attach a .txt file describing the steps I used to convert your data to a FLIM ome.tif using Fiji. I only tried with the .tif but it should work with any data that's open in Fiji. Once you have the tools setup it should only take a minute.

Can you have a try and see how it goes please. Hopefully this will be enough to evaluate FLIMfit with your data.

FijiFLIM.txt

[HagaiHargil]

@imunro I tried running the script but received the following MATLAB error:

Error using bfsave (line 113)
Java exception occurred:
java.lang.IllegalArgumentException
    at java.nio.Buffer.limit(Unknown Source)
    at loci.common.NIOFileHandle.buffer(NIOFileHandle.java:609)
    at loci.common.NIOFileHandle.writeSetup(NIOFileHandle.java:615)
    at loci.common.NIOFileHandle.write(NIOFileHandle.java:478)
    at loci.common.NIOFileHandle.write(NIOFileHandle.java:472)
    at loci.common.NIOFileHandle.write(NIOFileHandle.java:460)
    at loci.common.RandomAccessOutputStream.write(RandomAccessOutputStream.java:169)
    at loci.formats.tiff.TiffSaver.writeImageIFD(TiffSaver.java:537)
    at loci.formats.tiff.TiffSaver.writeImage(TiffSaver.java:397)
    at loci.formats.tiff.TiffSaver.writeImage(TiffSaver.java:273)
    at loci.formats.out.TiffWriter.saveBytes(TiffWriter.java:228)
    at loci.formats.out.OMETiffWriter.saveBytes(OMETiffWriter.java:195)
    at loci.formats.out.OMETiffWriter.saveBytes(OMETiffWriter.java:185)
    at loci.formats.FormatWriter.saveBytes(FormatWriter.java:124)
    at loci.formats.ImageWriter.saveBytes(ImageWriter.java:226)

Error in testOMETIFFGenerate (line 50)
bfsave(data, outputPath, 'metadata', metadata);

I loaded the data to memory using the h5reader function, and received a 4D stack arranged as [lifetime - x - y - time]. I added a sizet variable corresponding to the time step between each lifetime "step", but other than that no changes to the script were made. Would you happen to know what could be the problem? I might try to save the OME Tiff from the Python implementation at Bio-Format's website.

Thanks!

[imunro]

@HagaiHargil The script assumes that the data is in a 5D array with the timepoints in the last dimension. The dimensions are - x,y,Z,Channels,timepoints - in that order. as initialised in the script with sizeZ and sizeC ==1 data = int16(zeros(sizeX,sizeY,1,1,sizet)); Maybe that's the problem.

Please also note that sizet in the script is the number of timepoints. The interval in ps between timepoints is set here https://github.com/imunro/ic-converterM/blob/master/testOMETIFFGenerate.m#L91-L93

Ian

[liorgolgher]

Hi Ian,

Thanks to your detailed instructions, we have successfully prepared FLIMfit-readable OME-TIFF files using both options: Via FIJI and tiffcomment and using a Python implementation equivalent to the Matlab script you have shared. While xmlvalid complained about two issues with the tiffcomment file, FLIMfit had no problem loading it.

Expectedly FLIMfit offers some powerful tools through a very friendly user interface. Currently the lifetimes it estimates are in the order of 500-1000 ps, even when we fix the lifetime of a single fluorophore to those expected for FITC (~4.1 ns). I suspect that this problem has to do with our raw data. We will roll back to simpler ex-vivo experiments and continue from there.

Thank you very much @seanwarren @imunro and @dgault for resolving this issue!

Best, Lior

[seanwarren]

Hi Lior, glad to hear it's working for you - there are some tutorial screencasts at https://vimeo.com/flimfit to get you started with the analysis. If you have any problems give me or Ian a shout and we can help you out.

[imunro]

Hi Lior

Good to know that it's working. The xmlvalid failure is slightly worrying. Does it give any details? @seanwarren I did wonder if this data is more like Gated than TCSPC - any thoughts?

[liorgolgher]

Regrettably I did not save the log of xmlvalid, and we're just about to enter a week-long national holiday. I'll send you the log once I return but it didn't look too serious, probably a version mismatch of some sort.

[imunro]

Have a good holiday.

[liorgolgher]

OK, here's what xmlvalid has to say. As noted above, FLIMfit had no problem opening the file:

Parsing schema path http://www.openmicroscopy.org/Schemas/OME/2015-01/ome.xsd Validating FlimFitTestMac.ome.tif cvc-complex-type.2.4.a: Invalid content was found starting with element 'AnnotationRef'. One of '{"http://www.openmicroscopy.org/Schemas/ROI/2015-01":ROIRef, "http://www.openmicroscopy.org/Schemas/OME/2015-01":MicrobeamManipulationRef, "http://www.openmicroscopy.org/Schemas/SA/2015-01":AnnotationRef}' is expected. cvc-complex-type.2.4.a: Invalid content was found starting with element 'StructuredAnnotations'. One of '{"http://www.openmicroscopy.org/Schemas/OME/2015-01":Image, "http://www.openmicroscopy.org/Schemas/SA/2015-01":StructuredAnnotations, "http://www.openmicroscopy.org/Schemas/ROI/2015-01":ROI}' is expected. Error validating document: 2 errors found

[imunro]

Hi Lior

It's possible that there's a typo in the .txt file that I posted above. For the purposes of evaluating FLIMfit I guess that it's fine but if you decide to go to "production" then please send me an ome-tiff and I'll see if I can spot it.