search

bioimage-io / core-bioimage-io-python

Python libraries for loading, running and packaging bioimage.io models
https://bioimage-io.github.io/core-bioimage-io-python/
MIT License
28 stars 21 forks source link

Save 3D, 2D+c or 2D+t test and sample images correctly #210

Closed esgomezm closed 2 years ago

esgomezm commented 2 years ago

Hi! I'm converting the ZCDL4M 3D U-Net notebook with the bioimageio.core code and I see that 3D images are not properly stored. This also happened to StarDist as far as I remember when the images were 2D + channels. Could it be possible to correct it?

Thank you!

constantinpape commented 2 years ago

Hi @esgomezm, I think that this is a discrepancy with how tifs with more than 2 dimensions are written here and read in deepImagej/Fiji. And this is definitely something we want to fix. Currently, the code for writing the tif samples is here. So we would need to adapt this code so that it's compatible with deepIJ. But I don't really know how to test this easily, as I can read it back correctly in python, e.g. for the stardist example model:

import imageio
out = imageio.volread("sample_output_0.tif")
print(out.shape)
Out[3]: (512, 512, 33)
esgomezm commented 2 years ago

Hi @constantinpape I answered you by email as well. But we can discuss it here. The code I normally use with tiff files is the following one:

tifffile.imsave(file_name),
                    np.expand_dims(video, axis=[1, 2, -1]), # addapt the dimensions to tifffile ordering TZCYXS
                    resolution=(1 / pixel_size, 1 / pixel_size),
                    imagej=True,
                    metadata={'spacing': 1, 'unit': 'um'})

Storing the tiff like that shouldn't give you problems when recovering it from python. Could you test it?

Also, can use the very last version of the bioimageio.core in the notebook?

constantinpape commented 2 years ago

Thanks for the code snippet @esgomezm. I am working on using it now, see #212.

Also, can use the very last version of the bioimageio.core in the notebook?

Sorry, I don't quite understand what you mean, could you please clarify?

constantinpape commented 2 years ago

@esgomezm I think I have fixed the issues now and I have made a new release already that includes the changes: v0.4.10. If you switch to it in the zero cost notebooks the problems should be fixed.

constantinpape commented 2 years ago

This works now (I checked that the tifs can be read in Fiji and @carlosuc3m checked in deepIJ for an example model). @esgomezm the changes are already released in v0.4.10, so if you switch to this version in zero-cost the model export should work.

esgomezm commented 2 years ago

Hi!

I'm using v0.4.11 and the tiff export gives the following error with 3D data. Looking at the tifffile library, seems that the type needs to be specified in tifffile.write but I'm not 100% sure. Any idea?

[/usr/local/lib/python3.7/dist-packages/bioimageio/core/build_spec/build_model.py](https://localhost:8080/#) in write_im(path, im, axes, pixel_size)
    441         if np.dtype(im.dtype) == np.dtype("float64"):
    442             im = im.astype("float32")
--> 443         tifffile.imsave(path, im, imagej=True, resolution=resolution)
    444 
    445     sample_in_paths = []

[/usr/local/lib/python3.7/dist-packages/tifffile/tifffile.py](https://localhost:8080/#) in imwrite(file, data, shape, dtype, **kwargs)
    964 
    965     with TiffWriter(file, **tifargs) as tif:
--> 966         result = tif.write(data, shape, dtype, **kwargs)
    967     return result
    968 

[/usr/local/lib/python3.7/dist-packages/tifffile/tifffile.py](https://localhost:8080/#) in write(self, data, shape, dtype, photometric, planarconfig, extrasamples, volumetric, tile, contiguous, truncate, align, rowsperstrip, bitspersample, compression, predictor, subsampling, jpegtables, colormap, description, datetime, resolution, subfiletype, software, subifds, metadata, extratags, returnoffset, ijmetadata, compress)
   1768             if datadtypechar not in 'BHhf':
   1769                 raise ValueError(
-> 1770                     'the ImageJ format does not support data type '
   1771                     f'{datadtypechar!r}'
   1772                 )

ValueError: the ImageJ format does not support data type '?'

You can check the notebook here: https://github.com/esgomezm/ZeroCostDL4Mic/blob/master/Colab_notebooks/BioImage.io%20notebooks/U-Net_3D_ZeroCostDL4Mic_BioImageModelZoo_export.ipynb

FynnBe commented 2 years ago

ValueError: the ImageJ format does not support data type '?'

The "?" seems to come from https://github.com/blink1073/tifffile/blob/af9d70058b01f501b29cfe924a6463414c388480/tifffile/tifffile.py#L891 which means boolean data: https://numpy.org/doc/stable/reference/arrays.dtypes.html

seems unrelated to bioimageio.core: The input data should not have dtype boolean to be saved as "ImageJ Tiff"?? How do you deal with boolean arrays in ImageJ?

constantinpape commented 2 years ago

Thanks for looking into this @FynnBe. Indeed it looks like an issue with the data type. @esgomezm please check that the numpy array passed to https://github.com/bioimage-io/core-bioimage-io-python/blob/main/bioimageio/core/build_spec/build_model.py#L588-L589 has a "normal" datatype (i.e. not bool or some string datatype).

esgomezm commented 2 years ago

Solved! It was boolean datatype

carlosuc3m commented 2 years ago

I am reopening this issue because I noticed that some of the images (the ones with Z-dimension) are converted incorrectly from .npy to .tif. It seems that the X, Z and B coordinates are confused. Here I am writting a list of the models where I encountered this issue:

In addition to this, there are some models that do not contain a tif sample image and I would like to ask if it is possible to add one. The models are the followings

@constantinpape @FynnBe @esgomezm

constantinpape commented 2 years ago

Hi @carlosuc3m, I am a bit puzzled by this, since I tried the tif export functionality locally and it worked fine for me (i.e. the tifs can be loaded correctly in Fiji). So I am not sure how to fix this and also don't really have time to look into it right now. Maybe you could have a look at it? The function for saving the tifs is this one: https://github.com/bioimage-io/core-bioimage-io-python/blob/main/bioimageio/core/build_spec/build_model.py#L421-L450. If you manage to fix it I am happy to help out with updating the tifs in the models where they are wrong. (and we can of course add the tifs for the other models, but that only makes sense once we have a fully working converter.)

carlosuc3m commented 2 years ago

Okk, will do and will get back to this issue.

carlosuc3m commented 2 years ago

I just create a pull request ( https://github.com/bioimage-io/core-bioimage-io-python/pull/275 ) that should solve the issue. It was pretty straight forward to solve. The 2D image generation was also wrong but it had not created any problem because the c and b dimensions where the ones mixed and in the models all the images had only one channel and one batch. I am also linking the documentation in the tifffile project that justifies the change: https://github.com/cgohlke/tifffile/blob/master/tifffile/tifffile.py#L1348-L1357

constantinpape commented 2 years ago

I just create a pull request ( #275 ) that should solve the issue. It was pretty straight forward to solve. The 2D image generation was also wrong but it had not created any problem because the c and b dimensions where the ones mixed and in the models all the images had only one channel and one batch.

Unfortunately it seems to not be that straight forward, since your changes now break the tests because some of the images cannot be written any more. Could you please have another look?

constantinpape commented 2 years ago

Thanks for fixing this @carlosuc3m. I made a new release (0.5.5) including your changes.