Folder of 2D tifs

[VolkerH]

I'm trying to use this on a folder full of 2D tiff slices. All in all, I have 6 dimensions:

1. illumination direction
2. tile postion u
3. tile position v
4. z-slice
5. x
6. y

only x and y dimensions are contained in the individual tiff files in the folder. The code bombs out with a ShapeError. This should not be quite unexpected as during debugging I noticed that this happens in a section of code with a # FIX ME: ASSUMING THAT FILES ARE Z STACKS FOR NOW tag.

Is this something that is a fairly quick fix? I started looking into the code, but haven't quite gotten my head around how you pass around indexing and slicing objects.

This is the code snippet that produces the error:

pattern = {'u':'{d} x', 'w':'x {d}', 'd': 'C{d2}', 's': ' Z{d}'}
f = tifffolder.TiffFolder('C:/Users/Volker/Data/180903_UM_Test/tmp', pattern, axes ='uwds' )
print("excluded" , f.list_excluded())
print("files", f.files)
print("shape", f.shape)
print("pixel", f[0,0,0,0,0,0])

and here is what happens when trying to get to that first voxel:

BTW: I came across your tifffolder and imarispy repos while looking at LLSpy. As I'm trying to create some sort of streamlined workflow to collect files from a LaVision Biotech Ultramicroscope and convert them to Big Data Viewer format for stitching with Big Data Viewer both these projects seem to offer an elegant solution (without using Java). So thanks for making these public.

[tlambert03]

Hey, thanks for having a look.

glad you're playing around with it and hope that it can be useful for you! I'm not sure anyone else has really tried to use this besides me, so there will likely be some issues like this that need addressing... but i'm happy to try to modify the code (or accept PRs) to get it working for you if you think it's going to be a good match for your needs. The imarispy repo should also be "close" to working (particularly for BDV... less so for imaris without a modified h5py library).

for this problem of 2D images, it will be easiest to help if I can get a sample dataset to play with. Can you email me a dropbox link (or post here or use whatever method you prefer for sharing larger data)? It doesn't need to have all the data, but at least a few images across each dimension. I think it shouldn't be too hard, and it would obviously improve the library to handle this scenario.

[VolkerH]

Hi, thanks for your reply. I've put together a stripped-down dataset for you here: https://www.dropbox.com/s/xx3y93bo0qk4ail/forTalley.zip?dl=0 (I eliminated most of the Z-slices otherwise all of the structure is preserved).

The sample code is here. https://gist.github.com/VolkerH/b80d1a00173e3e011bc1ebcb46910d65

Note that in the comments I also point out some seperate issues, regarding the naming of the dictionary keys. These are probably mainly an issue of being more clear in the documentation that while some dictionary keys can be used for pattern matching, they can't be used to instantiate an object of the TiffFolder class.

[tlambert03]

thanks so much! give me a little time to work on it and I'll get back to you

[VolkerH]

I have to thank you!

[tlambert03]

ok, have initial support for 2D images ready for testing... and I fixed the problem where the z axis cannot be specified as "z" in the patterns. (however, for now, multi-character pattern keys will only work for non-axis dimensions (for instance... the two examples in the readme were both grabbing metadata in the names... rather than adding a new dimension to the dataset. subtle I know, and I need to make that clearer and fix it)...

I suspect more issues may pop up, but maybe give it a try again and let me know (please use the development branch of both tifffolder and imarispy... not master or conda branches). The code below works for me to convert your data to either a big-data-viewer dataset or an imaris file (note: the imaris file can be open by Fiji and BDV, but not Imaris yet due to some limitations i've run into in Imaris itself)

import tifffolder
import imarispy

folder = '~/Desktop/forTalley'
pattern = {'u':'{d} x', 'w':'x {d}', 'd': 'C{d2}', 'z': ' Z{d4}'}
f = tifffolder.TiffFolder(folder, pattern, axes='uwdzyx')
f.shape == f.asarray().shape  # TRUE
imarispy.np_to_ims(f[0, 0], 'imaris_test')
# OR
imarispy.np_to_bdv(f[0, 0], 'bdv_test')

thanks a lot for the helpful gists, and the data.

[VolkerH]

Thank you very much for your help, that was quick. It works very nicely, thank you. I get some warnings when saving to BDV, but the resulting files open just fine in BDV

I have something I can work with now. Thanks again !

[tlambert03]

yeah, those warnings are due to the (somewhat clunky) tifffolder.AxesArray object, which is a very thin subclass of a np.ndarray that I hoped would be able to keep better track of which axis was which following operations like indexing, transposing, etc... But it may end up just being annoying.

If you want to get rid of those, and just ditch the AxesArray at any point, you can just convert the data returned by tifffolder to a np.ndarray, e.g. f = np.array(f)

glad it's working for you though. feel free to let me know if you run into additional issues.