Importing Operetta data using Archived files

[jamila-griffith]

Hi I am just wondering if we can use this script to import Operetta data which is archived instead of Export.

[NicoKiaru]

I wish! Unfortunately that's not possible. You need to import in Harmony and re-export... We totally rely on the bio-formats library (https://docs.openmicroscopy.org/bio-formats/6.10.0/supported-formats.html), and it can't read the archive.

[diligentgiantsquid]

Hi @NicoKiaru and @lacan! Thanks again for your continuing work on this tool, it's super useful to us.
I wanted to ask if your position on this has changed, since Harmony v5.1? The new /hs format for the data, including archives, is now bioformats-readable. I tried pointing OperettaImporter to this new archive format, and it seems to read the metadata correctly, and generate the converted files as expected, only the images are blank. Is this what you'd currently expect?
I guess I wanted to ask as the Export function is now marked 'soon to be retired' in the new version of Harmony :-S Thanks

[NicoKiaru]

Hi @diligentgiantsquid,

The new /hs format for the data, including archives, is now bioformats-readable

That's good news

only the images are blank

It's less good news.

So the question is whether the issue is on this plugin side or on the bio-formats side. Could you share a small typical archive (for instance on Zenodo?). We could have a look. Something simple you can test is to try to open your dataset directly in Fiji, not using the Operetta importer. If the images are blank, then it means it's a bio-formats issue.

ping @romainGuiet

[diligentgiantsquid]

Many thanks @NicoKiaru for your quick reply. I wrote a new archive of a small example dataset, and uploaded this hs-format archive to Zenodo, here https://zenodo.org/doi/10.5281/zenodo.10776784

I checked it opens OK in FIJI using the normal Bioformats importer which it does. As before appears blank with OperettaImporter.

[NicoKiaru]

Thanks for the dataset!

I think it should be doable. The current issue comes from the pattern matching:

Files are now named: "r03c03f01p01-ch01t01.tiff" while we try to match a different pattern:

https://github.com/BIOP/ijp-operetta-importer/blob/bf35b2559b162266445e28167a3c6e093fb10080/src/main/java/ch/epfl/biop/operetta/utils/HyperRange.java#L42

i won't have time to look at this in the following two weeks, but I'll try to get back to you after that or maybe @lacan will want to check it before.

[diligentgiantsquid]

Many thanks for considering it - if it's possible this would be a great enhancement. Best wishes, Huw

[NicoKiaru]

@diligentgiantsquid I've made it work with the help of your sample dataset: https://github.com/BIOP/ijp-operetta-importer/commit/6c3f5e6c4f6060d22832de965ba3d98ad8cfffa5

I've put the current SNAPSHOT operetta importer on the PTBIOP update site. Let us know if it works, and if so I'll make a cleaner release.

[diligentgiantsquid]

Amazing! That works perfectly in my hands - I tried with a few different archives (some with fusing the fields etc, but I don't have any timelapses to hand to try). Let me know if it's useful to have different dimensions of measurements for testing and I can try to generate some.
That's brilliant, thank you so much :-D

[NicoKiaru]

Great, I'll make a release and close the issue. Let's deal with the potential T issue when it happens, if it happens

[jamila-griffith]

It looks like the issue has already been closed. Just wondering did you update the PTBIOP or used a different way to solve this?

[NicoKiaru]

@jamila-griffith it's just an update of the PTBIOP update site. Normally if you update Fiji, it should work. Note that I'm not sure all archives will work. But for sure it works for the archive shared by @diligentgiantsquid