Built-in writer crashes when trying to save multiple pyramid layers to zarr

jni commented 4 years ago

🐛 Bug

When trying to save multiple pyramidal image layers to file.zarr, I get the following traceback:

Traceback (most recent call last):
  File "/home/jni/miniconda3/envs/all/lib/python3.8/site-packages/napari_plugin_engine/hooks.py", line 404, in _call_plugin
    return implementation(*_args)
  File "/home/jni/miniconda3/envs/all/lib/python3.8/site-packages/napari_plugin_engine/implementation.py", line 66, in __call__
    return self.function(*args)
  File "/home/jni/projects/napari/napari/plugins/_builtins.py", line 90, in napari_write_image
    imsave(path, data)
  File "/home/jni/projects/napari/napari/utils/io.py", line 30, in imsave
    tifffile.imsave(filename, data)
  File "/home/jni/miniconda3/envs/all/lib/python3.8/site-packages/tifffile/tifffile.py", line 655, in imwrite
    return tif.save(data, shape, dtype, **kwargs)
  File "/home/jni/miniconda3/envs/all/lib/python3.8/site-packages/tifffile/tifffile.py", line 1043, in save
    data = numpy.asarray(data, byteorder + data.dtype.char, 'C')
AttributeError: 'list' object has no attribute 'dtype'

To Reproduce

Steps to reproduce the behavior:

Run napari https://s3.embassy.ebi.ac.uk/idr/zarr/v0.1/6001240.zarr
Click File -> Save all layers
Use the filename image.zarr in an arbitrary folder

Expected behavior

napari should not have thought that tifffile could handle this task. ;)

Environment

napari: 0.3.2rc0+1.g7e5d8ca1
Platform: Linux-4.15.0-29-generic-x86_64-with-glibc2.10
Python: 3.8.2 (default, Mar 26 2020, 15:53:00) [GCC 7.3.0]
Qt: 5.14.2
PyQt5: 5.14.2
NumPy: 1.19.0rc1
SciPy: 1.4.1
Dask: 2.15.0
VisPy: 0.6.4

GL version: 3.0 Mesa 19.2.8
MAX_TEXTURE_SIZE: 16384

Plugins:
- napari-plugin-engine: 0.1.4
- ome_zarr: 0.0.7
- svg: 0.1.2

dstansby commented 1 year ago

Looks like that file doesn't exist any more - could you link to another file to reproduce this with?

jni commented 1 year ago

👋 ohai!

This is my source of truth for where the latest IDR data lives. 😊

https://github.com/ome/napari-ome-zarr#usage

psobolewskiPhD commented 1 year ago

I tried: napari "https://uk1s3.embassy.ebi.ac.uk/idr/zarr/v0.3/9836842.zarr/" With fresh env, just builtins, I get:

ReaderPluginError: Tried to read https://uk1s3.embassy.ebi.ac.uk/idr/zarr/v0.3/9836842.zarr/ with plugin napari, because it was associated with that file extension/because it is the only plugin capable of reading that path, but it gave an error. Try associating a different plugin or installing a different plugin for this kind of file.

After pip install napari-ome-zarr it works.

Doing Save all layers... setting: builtins folder, name image.zarr I get a different error:

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
~/Dev/miniforge3/envs/napari-10/lib/python3.10/site-packages/napari/_qt/menus/file_menu.py in ?()
---> 81                 'slot': lambda: window._qt_viewer._save_layers_dialog(
     82                     selected=False
     83                 ),

~/Dev/miniforge3/envs/napari-10/lib/python3.10/site-packages/napari/_qt/qt_viewer.py in ?(self=<napari._qt.qt_viewer.QtViewer(0x600002aac360)>, selected=False)
    593             writer = _npe2_decode_selected_filter(
    594                 ext_str, selected_filter, writers
    595             )
    596             with warnings.catch_warnings(record=True) as wa:
--> 597                 saved = self.viewer.layers.save(
    598                     filename, selected=selected, _writer=writer
    599                 )
    600                 logging.debug('Saved %s', saved)

~/Dev/miniforge3/envs/napari-10/lib/python3.10/site-packages/napari/components/layerlist.py in ?(self=[<Image layer 'Cam2-T1' at 0x2947cdc60>, <Image ...t 0x294829de0>, <Image layer '3' at 0x296a46cb0>], path='/Users/piotrsobolewski/Downloads/image.zarr', selected=False, plugin=None, _writer=WriterContribution(command='napari.write_directo...ame_extensions=[], display_name='Save to Folder'))
    437         if not layers:
    438             warnings.warn(msg)
    439             return []
    440 
--> 441         return save_layers(path, layers, plugin=plugin, _writer=_writer)

~/Dev/miniforge3/envs/napari-10/lib/python3.10/site-packages/napari/plugins/io.py in ?(path='/Users/piotrsobolewski/Downloads/image.zarr', layers=[<Image layer 'Cam2-T1'>, <Image layer 'Cam1-T2'>, <Image layer '2'>, <Image layer '3'>], plugin=None, _writer=WriterContribution(command='napari.write_directo...ame_extensions=[], display_name='Save to Folder'))
    221     """
    222 
    223     writer_name = ''
    224     if len(layers) > 1:
--> 225         written, writer_name = _write_multiple_layers_with_plugins(
    226             path, layers, plugin_name=plugin, _writer=_writer
    227         )
    228     elif len(layers) == 1:

~/Dev/miniforge3/envs/napari-10/lib/python3.10/site-packages/napari/plugins/io.py in ?(path='/Users/piotrsobolewski/Downloads/image.zarr', layers=[<Image layer 'Cam2-T1'>, <Image layer 'Cam1-T2'>, <Image layer '2'>, <Image layer '3'>], plugin_name=None, _writer=WriterContribution(command='napari.write_directo...ame_extensions=[], display_name='Save to Folder'))
    327         Name of the plugin selected to write the data.
    328     """
    329 
    330     # Try to use NPE2 first
--> 331     written_paths, writer_name = _npe2.write_layers(
    332         path, layers, plugin_name, _writer
    333     )
    334     if written_paths or writer_name:

~/Dev/miniforge3/envs/napari-10/lib/python3.10/site-packages/napari/plugins/_npe2.py in ?(path='/Users/piotrsobolewski/Downloads/image.zarr', layers=[<Image layer 'Cam2-T1'>, <Image layer 'Cam1-T2'>, <Image layer '2'>, <Image layer '3'>], plugin_name=None, writer=WriterContribution(command='napari.write_directo...ame_extensions=[], display_name='Save to Folder'))
    113             return paths, writer.plugin_name
    114 
    115     n = sum(ltc.max() for ltc in writer.layer_type_constraints())
    116     args = (path, *layer_data[0][:2]) if n <= 1 else (path, layer_data)
--> 117     res = writer.exec(args=args)
    118     if isinstance(
    119         res, str
    120     ):  # pragma: no cover # it shouldn't be... bad plugin.

~/Dev/miniforge3/envs/napari-10/lib/python3.10/site-packages/npe2/manifest/utils.py in ?(self=WriterContribution(command='napari.write_directo...ame_extensions=[], display_name='Save to Folder'), args=('/Users/piotrsobolewski/Downloads/image.zarr', [(<MultiScaleData at 0x294828070. 6 levels, 'uint1...), (480, 480), (240, 240), (120, 120), (60, 60))>, {'affine': <class 'numpy.ndarray'> (3, 3) float64, 'attenuation': 0.05, 'blending': 'translucent_no_depth', 'colormap': '[unnamed colormap 0]', 'contrast_limits': [198.0, 5090.0], 'custom_interpolation_kernel_2d': <class 'numpy.ndarray'> (1, 1) float32, 'depiction': 'volume', 'experimental_clipping_planes': [], 'gamma': 1, 'interpolation2d': 'nearest', ...}, 'image'), (<MultiScaleData at 0x296a90f70. 6 levels, 'uint1...), (480, 480), (240, 240), (120, 120), (60, 60))>, {'affine': <class 'numpy.ndarray'> (3, 3) float64, 'attenuation': 0.05, 'blending': 'additive', 'colormap': '[unnamed colormap 1]', 'contrast_limits': [198.0, 5736.0], 'custom_interpolation_kernel_2d': <class 'numpy.ndarray'> (1, 1) float32, 'depiction': 'volume', 'experimental_clipping_planes': [], 'gamma': 1, 'interpolation2d': 'nearest', ...}, 'image'), (<MultiScaleData at 0x296a45990. 6 levels, 'uint1...), (480, 480), (240, 240), (120, 120), (60, 60))>, {'affine': <class 'numpy.ndarray'> (3, 3) float64, 'attenuation': 0.05, 'blending': 'additive', 'colormap': '[unnamed colormap 2]', 'contrast_limits': [196.0, 5312.0], 'custom_interpolation_kernel_2d': <class 'numpy.ndarray'> (1, 1) float32, 'depiction': 'volume', 'experimental_clipping_planes': [], 'gamma': 1, 'interpolation2d': 'nearest', ...}, 'image'), (<MultiScaleData at 0x29795cbe0. 6 levels, 'uint1...), (480, 480), (240, 240), (120, 120), (60, 60))>, {'affine': <class 'numpy.ndarray'> (3, 3) float64, 'attenuation': 0.05, 'blending': 'additive', 'colormap': '[unnamed colormap 3]', 'contrast_limits': [194.0, 4128.0], 'custom_interpolation_kernel_2d': <class 'numpy.ndarray'> (1, 1) float32, 'depiction': 'volume', 'experimental_clipping_planes': [], 'gamma': 1, 'interpolation2d': 'nearest', ...}, 'image')]), kwargs={}, _registry=None)
     59     ) -> R:
     60         if kwargs is None:
     61             kwargs = {}
     62         reg = _registry or kwargs.pop("_registry", None)
---> 63         return self.get_callable(reg)(*args, **kwargs)

~/Dev/miniforge3/envs/napari-10/lib/python3.10/site-packages/napari_builtins/io/_write.py in ?(path='/Users/piotrsobolewski/Downloads/image.zarr', layer_data=[(<MultiScaleData at 0x294828070. 6 levels, 'uint1...), (480, 480), (240, 240), (120, 120), (60, 60))>, {'affine': <class 'numpy.ndarray'> (3, 3) float64, 'attenuation': 0.05, 'blending': 'translucent_no_depth', 'colormap': '[unnamed colormap 0]', 'contrast_limits': [198.0, 5090.0], 'custom_interpolation_kernel_2d': <class 'numpy.ndarray'> (1, 1) float32, 'depiction': 'volume', 'experimental_clipping_planes': [], 'gamma': 1, 'interpolation2d': 'nearest', ...}, 'image'), (<MultiScaleData at 0x296a90f70. 6 levels, 'uint1...), (480, 480), (240, 240), (120, 120), (60, 60))>, {'affine': <class 'numpy.ndarray'> (3, 3) float64, 'attenuation': 0.05, 'blending': 'additive', 'colormap': '[unnamed colormap 1]', 'contrast_limits': [198.0, 5736.0], 'custom_interpolation_kernel_2d': <class 'numpy.ndarray'> (1, 1) float32, 'depiction': 'volume', 'experimental_clipping_planes': [], 'gamma': 1, 'interpolation2d': 'nearest', ...}, 'image'), (<MultiScaleData at 0x296a45990. 6 levels, 'uint1...), (480, 480), (240, 240), (120, 120), (60, 60))>, {'affine': <class 'numpy.ndarray'> (3, 3) float64, 'attenuation': 0.05, 'blending': 'additive', 'colormap': '[unnamed colormap 2]', 'contrast_limits': [196.0, 5312.0], 'custom_interpolation_kernel_2d': <class 'numpy.ndarray'> (1, 1) float32, 'depiction': 'volume', 'experimental_clipping_planes': [], 'gamma': 1, 'interpolation2d': 'nearest', ...}, 'image'), (<MultiScaleData at 0x29795cbe0. 6 levels, 'uint1...), (480, 480), (240, 240), (120, 120), (60, 60))>, {'affine': <class 'numpy.ndarray'> (3, 3) float64, 'attenuation': 0.05, 'blending': 'additive', 'colormap': '[unnamed colormap 3]', 'contrast_limits': [194.0, 4128.0], 'custom_interpolation_kernel_2d': <class 'numpy.ndarray'> (1, 1) float32, 'depiction': 'volume', 'experimental_clipping_planes': [], 'gamma': 1, 'interpolation2d': 'nearest', ...}, 'image')])
    319                 shutil.move(os.path.join(tmp, fname), path)
    320     except Exception:
    321         if not already_existed:
    322             shutil.rmtree(path, ignore_errors=True)
--> 323         raise
    324     return written

~/Dev/miniforge3/envs/napari-10/lib/python3.10/site-packages/npe2/io_utils.py in ?(path='/Users/piotrsobolewski/Downloads/image.zarr/tmpqf05am0x/Cam2-T1', layer_data=[(<MultiScaleData at 0x294828070. 6 levels, 'uint1...), (480, 480), (240, 240), (120, 120), (60, 60))>, {'affine': <class 'numpy.ndarray'> (3, 3) float64, 'attenuation': 0.05, 'blending': 'translucent_no_depth', 'colormap': '[unnamed colormap 0]', 'contrast_limits': [198.0, 5090.0], 'custom_interpolation_kernel_2d': <class 'numpy.ndarray'> (1, 1) float32, 'depiction': 'volume', 'experimental_clipping_planes': [], 'gamma': 1, 'interpolation2d': 'nearest', ...}, 'image')], plugin_name='napari')
    101     ------
    102     ValueError
    103         If no suitable writers are found.
    104     """
--> 105     return _write(path, layer_data, plugin_name=plugin_name)

~/Dev/miniforge3/envs/napari-10/lib/python3.10/site-packages/npe2/io_utils.py in ?(path='/Users/piotrsobolewski/Downloads/image.zarr/tmpqf05am0x/Cam2-T1', layer_data=[(<MultiScaleData at 0x294828070. 6 levels, 'uint1...), (480, 480), (240, 240), (120, 120), (60, 60))>, {'affine': <class 'numpy.ndarray'> (3, 3) float64, 'attenuation': 0.05, 'blending': 'translucent_no_depth', 'colormap': '[unnamed colormap 0]', 'contrast_limits': [198.0, 5090.0], 'custom_interpolation_kernel_2d': <class 'numpy.ndarray'> (1, 1) float32, 'depiction': 'volume', 'experimental_clipping_planes': [], 'gamma': 1, 'interpolation2d': 'nearest', ...}, 'image')], plugin_name='napari', return_writer=False, _pm=<npe2._plugin_manager.PluginManager object>)
    227     # Writers that take at most one layer must use the single-layer api.
    228     # Otherwise, they must use the multi-layer api.
    229     n = sum(ltc.max() for ltc in writer.layer_type_constraints())
    230     args = (new_path, *_layer_tuples[0][:2]) if n <= 1 else (new_path, _layer_tuples)
--> 231     res = writer.exec(args=args)
    232 
    233     # napari_get_writer-style writers don't always return a list
    234     # though strictly speaking they should?

~/Dev/miniforge3/envs/napari-10/lib/python3.10/site-packages/npe2/manifest/utils.py in ?(self=WriterContribution(command='napari.write_image',...'.sgi', '.stk', '.tga'], display_name='lossless'), args=('/Users/piotrsobolewski/Downloads/image.zarr/tmpqf05am0x/Cam2-T1.tif', <MultiScaleData at 0x294828070. 6 levels, 'uint1...), (480, 480), (240, 240), (120, 120), (60, 60))>, {'affine': <class 'numpy.ndarray'> (3, 3) float64, 'attenuation': 0.05, 'blending': 'translucent_no_depth', 'colormap': '[unnamed colormap 0]', 'contrast_limits': [198.0, 5090.0], 'custom_interpolation_kernel_2d': <class 'numpy.ndarray'> (1, 1) float32, 'depiction': 'volume', 'experimental_clipping_planes': [], 'gamma': 1, 'interpolation2d': 'nearest', ...}), kwargs={}, _registry=None)
     59     ) -> R:
     60         if kwargs is None:
     61             kwargs = {}
     62         reg = _registry or kwargs.pop("_registry", None)
---> 63         return self.get_callable(reg)(*args, **kwargs)

~/Dev/miniforge3/envs/napari-10/lib/python3.10/site-packages/napari_builtins/io/_write.py in ?(path='/Users/piotrsobolewski/Downloads/image.zarr/tmpqf05am0x/Cam2-T1.tif', data=<MultiScaleData at 0x294828070. 6 levels, 'uint1...), (480, 480), (240, 240), (120, 120), (60, 60))>, meta={'affine': <class 'numpy.ndarray'> (3, 3) float64, 'attenuation': 0.05, 'blending': 'translucent_no_depth', 'colormap': '[unnamed colormap 0]', 'contrast_limits': [198.0, 5090.0], 'custom_interpolation_kernel_2d': <class 'numpy.ndarray'> (1, 1) float32, 'depiction': 'volume', 'experimental_clipping_planes': [], 'gamma': 1, 'interpolation2d': 'nearest', ...})
    120         path += '.tif'
    121         ext = '.tif'
    122 
    123     if ext in imsave_extensions():
--> 124         imsave(path, data)
    125         return path
    126 
    127     return None

~/Dev/miniforge3/envs/napari-10/lib/python3.10/site-packages/napari/utils/io.py in ?(filename='/Users/piotrsobolewski/Downloads/image.zarr/tmpqf05am0x/Cam2-T1.tif', data=<MultiScaleData at 0x294828070. 6 levels, 'uint1...), (480, 480), (240, 240), (120, 120), (60, 60))>)
     26     # Save screenshot image data to output file
     27     if ext in [".png"]:
     28         imsave_png(filename, data)
     29     elif ext in [".tif", ".tiff"]:
---> 30         imsave_tiff(filename, data)
     31     else:
     32         import imageio.v3 as iio
     33 

~/Dev/miniforge3/envs/napari-10/lib/python3.10/site-packages/napari/utils/io.py in ?(filename='/Users/piotrsobolewski/Downloads/image.zarr/tmpqf05am0x/Cam2-T1.tif', data=<MultiScaleData at 0x294828070. 6 levels, 'uint1...), (480, 480), (240, 240), (120, 120), (60, 60))>)
     96 
     97     if compression_instead_of_compress:
     98         # 'compression' scheme is more complex. See:
     99         # https://forum.image.sc/t/problem-saving-generated-labels-in-cellpose-napari/54892/8
--> 100         tifffile.imwrite(filename, data, compression=('zlib', 1))
    101     else:  # older version of tifffile since 2021.6.6  this is deprecated
    102         tifffile.imwrite(filename, data, compress=1)

~/Dev/miniforge3/envs/napari-10/lib/python3.10/site-packages/tifffile/tifffile.py in ?(file='/Users/piotrsobolewski/Downloads/image.zarr/tmpqf05am0x/Cam2-T1.tif', data=<MultiScaleData at 0x294828070. 6 levels, 'uint1...), (480, 480), (240, 240), (120, 120), (60, 60))>, bigtiff=False, byteorder=None, imagej=False, ome=None, shaped=None, append=False, shape=None, dtype=None, photometric=None, planarconfig=None, extrasamples=None, volumetric=False, tile=None, rowsperstrip=None, bitspersample=None, compression=('zlib', 1), compressionargs=None, predictor=None, subsampling=None, jpegtables=None, colormap=None, description=None, datetime=None, resolution=None, resolutionunit=None, subfiletype=None, software=None, metadata={}, extratags=None, contiguous=False, truncate=False, align=None, maxworkers=None, returnoffset=False)
   1247         imagej=imagej,
   1248         ome=ome,
   1249         shaped=shaped,
   1250     ) as tif:
-> 1251         result = tif.write(
   1252             data,
   1253             shape=shape,
   1254             dtype=dtype,

~/Dev/miniforge3/envs/napari-10/lib/python3.10/site-packages/tifffile/tifffile.py in ?(self=<tifffile.TiffWriter 'Cam2-T1.tif'>, data=<MultiScaleData at 0x294828070. 6 levels, 'uint1...), (480, 480), (240, 240), (120, 120), (60, 60))>, shape=None, dtype=None, photometric=None, planarconfig=None, extrasamples=None, volumetric=False, tile=None, rowsperstrip=None, bitspersample=None, compression=('zlib', 1), compressionargs=None, predictor=None, subsampling=None, jpegtables=None, colormap=None, description=None, datetime=None, resolution=None, resolutionunit=None, subfiletype=None, software=None, subifds=None, metadata={}, extratags=None, contiguous=False, truncate=False, align=None, maxworkers=None, returnoffset=False)
   1939         else:
   1940             # array-like
   1941             # must be C-contiguous numpy array of TIFF byteorder
   1942             if hasattr(data, 'dtype'):
-> 1943                 dataarray = numpy.asarray(
   1944                     data, byteorder + data.dtype.char, 'C'
   1945                 )
   1946             else:

TypeError: MultiScaleData.__array__() takes 1 positional argument but 2 were given

napari / napari