search

AllenCellModeling / aicsimageio

Image Reading, Metadata Conversion, and Image Writing for Microscopy Images in Python
https://allencellmodeling.github.io/aicsimageio
Other
205 stars 51 forks source link

Loading CZI with multiple scenes where every scene is a mosaic fails with IndexError #257

Closed sebi06 closed 3 years ago

sebi06 commented 3 years ago

System and Software

Description

Testfile: https://www.dropbox.com/s/tr0yxcaf6y4oeit/W96_B2%2BB4_S%3D2_T%3D1%3DZ%3D1_C%3D1_Tile%3D5x9.czi?dl=0

2 seperate scenes with 5x9 tiles each:

image

When I run the following code:

aics_img = AICSImage(testfilename)
aics_img.shape

I get the following error output:

Traceback (most recent call last):
  File "C:\ProgramData\Anaconda3\envs\imageanalysis_czi\lib\site-packages\IPython\core\interactiveshell.py", line 3437, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-1-ef8440d796b6>", line 1, in <module>
    aics_img.shape
  File "C:\ProgramData\Anaconda3\envs\imageanalysis_czi\lib\site-packages\aicsimageio\aics_image.py", line 489, in shape
    return self.xarray_dask_data.shape
  File "C:\ProgramData\Anaconda3\envs\imageanalysis_czi\lib\site-packages\aicsimageio\aics_image.py", line 370, in xarray_dask_data
    self.reader.mosaic_xarray_dask_data
  File "C:\ProgramData\Anaconda3\envs\imageanalysis_czi\lib\site-packages\aicsimageio\readers\reader.py", line 382, in mosaic_xarray_dask_data
    self._mosaic_xarray_dask_data = self._get_stitched_dask_mosaic()
  File "C:\ProgramData\Anaconda3\envs\imageanalysis_czi\lib\site-packages\aicsimageio\readers\czi_reader.py", line 735, in _get_stitched_dask_mosaic
    return self._construct_mosaic_xarray(self.dask_data)
  File "C:\ProgramData\Anaconda3\envs\imageanalysis_czi\lib\site-packages\aicsimageio\readers\czi_reader.py", line 688, in _construct_mosaic_xarray
    stitched = self._stitch_tiles(
  File "C:\ProgramData\Anaconda3\envs\imageanalysis_czi\lib\site-packages\aicsimageio\readers\czi_reader.py", line 668, in _stitch_tiles
    ans[tuple(ans_indexes)] = data[tuple(data_indexes)]
  File "C:\ProgramData\Anaconda3\envs\imageanalysis_czi\lib\site-packages\dask\array\core.py", line 1734, in __getitem__
    index2 = normalize_index(index, self.shape)
  File "C:\ProgramData\Anaconda3\envs\imageanalysis_czi\lib\site-packages\dask\array\slicing.py", line 911, in normalize_index
    check_index(i, d)
  File "C:\ProgramData\Anaconda3\envs\imageanalysis_czi\lib\site-packages\dask\array\slicing.py", line 981, in check_index
    raise IndexError(
IndexError: Index is not smaller than dimension 1 >= 1
evamaxfield commented 3 years ago

It's always either floating point errors or index errors 😅. I will give this a look soon...

Fair warning, it may not be until next week.

alpha-1-centauri commented 3 years ago

Same issue with getting data from Z-stack czi files on MacOS 10.15.7 and Python 3.8.

evamaxfield commented 3 years ago

Same issue with getting data from Z-stack czi files on MacOS 10.15.7 and Python 3.8.

Hey @AdamSya could you provide your traceback as well? And your test file if possible. Would love to see if its the exact same issue considering Z-Stacks are rarely (i think) used for mosaics?

sebi06 commented 3 years ago

HI @JacksonMaxfield

if I look at all ongoing projects (here at ZEISS) and what we get as feedback I would really not say the the combination mosaic and z-stacks is really rare. Especially in recent times such CZI become increasingly popular.

I uploaded some for CZIs with various dimensions. Feel free to keep all of them for testing and let me know, if you need more. My simulated microscope is really good at creating any arbitrary CZI ... :-)

S=2_4x2_T=2=Z=3_CH=2.czi: https://www.dropbox.com/s/0w24f4090sw6y46/S%3D2_4x2_T%3D2%3DZ%3D3_CH%3D2.czi?dl=0

S=3_1Pos_2Mosaic_T=2=Z=3_CH=2.czi: https://www.dropbox.com/s/u9yjhg8u5gm6duz/S%3D3_1Pos_2Mosaic_T%3D2%3DZ%3D3_CH%3D2.czi?dl=0

S=2_3x3_T=3_Z=4_CH=2.czi https://www.dropbox.com/s/1eegbku3zxlmde7/S%3D2_3x3_T%3D3_Z%3D4_CH%3D2.czi?dl=0

S=1_HE_Slide_RGB.czi https://www.dropbox.com/s/5guj227wt7spk1l/S%3D1_HE_Slide_RGB.czi?dl=0

And last (but not least) I created a really "as nasty as possible" CZI:

W96_S=2_P=2_T=2_Z=3_CH=2_LSM=DAPI_WF=GFP_MixedMode.czi: https://www.dropbox.com/s/42hoi2ee4aa0tsx/W96_S%3D2_P%3D2_T%3D2_Z%3D3_CH%3D2_LSM%3DDAPI_WF%3DGFP_MixedMode.czi?dl=0

evamaxfield commented 3 years ago

Thanks @sebi06!

evamaxfield commented 3 years ago

Question for you both: how urgent is this to fix?

My timeline for looking at this is on the order of weeks from now. Got some other stuff to do in my free time after work :sweat_smile: that said, if either of you would like to take a look at this issue please feel free and make a PR. I will happily review.

toloudis commented 3 years ago

I'll spend a little time today looking into this one

alpha-1-centauri commented 3 years ago

Apologies for the late response @JacksonMaxfield, also got held up with heaps of work this week. No rush - not a priority at the moment. Just thought I'd point out it's not an isolated error

Example file: https://transfer.sh/19WYSww/CR4-G-plate-2021-06-08-G2.czi

Code:

from aicsimageio import AICSImage stitched_image = AICSImage("/Project_dir/CR4-G-plate-2021-06-08-D6.czi") stitched_image.dims

Traceback:

Traceback (most recent call last): File "/Project_dir/test.py", line 16, in stitched_image.dims File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/aicsimageio/aics_image.py", line 501, in dims dims=self.xarray_dask_data.dims, shape=self.shape File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/aicsimageio/aics_image.py", line 370, in xarray_dask_data self.reader.mosaic_xarray_dask_data File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/aicsimageio/readers/reader.py", line 382, in mosaic_xarray_dask_data self._mosaic_xarray_dask_data = self._get_stitched_dask_mosaic() File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/aicsimageio/readers/czi_reader.py", line 735, in _get_stitched_dask_mosaic return self._construct_mosaic_xarray(self.dask_data) File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/aicsimageio/readers/czi_reader.py", line 688, in _construct_mosaic_xarray stitched = self._stitch_tiles( File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/aicsimageio/readers/czi_reader.py", line 668, in _stitch_tiles ans[tuple(ans_indexes)] = data[tuple(data_indexes)] File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/dask/array/core.py", line 1734, in getitem index2 = normalize_index(index, self.shape) File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/dask/array/slicing.py", line 911, in normalize_index check_index(i, d) File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/dask/array/slicing.py", line 981, in check_index raise IndexError( IndexError: Index is not smaller than dimension 1 >= 1

toloudis commented 3 years ago

Hopefully we have a good fix in #260

evamaxfield commented 3 years ago

This is now resolved and fixed in v4.0.2.

pip install aicsimageio --upgrade or pip install aicsimageio==4.0.2