Opening multi-4GB-OME-TIF file from MicroManager fails

Hi Christoph @cgohlke ,

you saw it already on image.sc. I'm experiencing issues when opening a multi-4GB-OME-TIF file in napari and apparently the issue is related to tifffile. I'm receiving the error below. Can you give advice of how to open such a file with python and/or napari?

20:10:44 WARNING <tifffile.TiffTag 270 @66911> coercing invalid ASCII to bytes
20:10:52 WARNING <tifffile.TiffTag 270 @66911> coercing invalid ASCII to bytes
20:12:27 WARNING <tifffile.TiffTag 270 @66911> coercing invalid ASCII to bytes
AICSImageIO: Reader will load image in-memory: False
20:12:35 WARNING <tifffile.TiffTag 270 @66911> coercing invalid ASCII to bytes
20:12:42 WARNING <tifffile.TiffTag 270 @66911> coercing invalid ASCII to bytes
20:12:50 WARNING <tifffile.TiffTag 270 @66911> coercing invalid ASCII to bytes
20:12:58 WARNING <tifffile.TiffTag 270 @66911> coercing invalid ASCII to bytes
20:12:58 WARNING <tifffile.TiffTag 270 @66911> coercing invalid ASCII to bytes
20:12:58 WARNING <tifffile.TiffTag 270 @66911> coercing invalid ASCII to bytes
20:12:58 WARNING <tifffile.TiffTag 270 @66911> coercing invalid ASCII to bytes
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
~/opt/anaconda3/envs/bio_39/lib/python3.9/site-packages/napari/_qt/qt_viewer.py in dropEvent(self=<napari._qt.qt_viewer.QtViewer object>, event=<PyQt5.QtGui.QDropEvent object>)
   1089                 readers=readers,
   1090             )
-> 1091             self._get_and_try_preferred_reader(
        self._get_and_try_preferred_reader = <bound method QtViewer._get_and_try_preferred_reader of <napari._qt.qt_viewer.QtViewer object at 0x198333820>>
        readerDialog = <napari._qt.dialogs.qt_reader_dialog.QtReaderDialog object at 0x1e6e283a0>
        readers = {'builtins': 'builtins', 'aicsimageio-in-memory': 'aicsimageio-in-memory', 'aicsimageio-out-of-memory': 'aicsimageio-out-of-memory'}
        error_message = ''
   1092                 readerDialog, readers, error_message
   1093             )

~/opt/anaconda3/envs/bio_39/lib/python3.9/site-packages/napari/_qt/qt_viewer.py in _get_and_try_preferred_reader(self=<napari._qt.qt_viewer.QtViewer object>, readerDialog=<napari._qt.dialogs.qt_reader_dialog.QtReaderDialog object>, readers={'aicsimageio-in-memory': 'aicsimageio-in-memory', 'aicsimageio-out-of-memory': 'aicsimageio-out-of-memory', 'builtins': 'builtins'}, error_message='')
   1156             display_name, persist_choice = choice
   1157             plugin_name = readers[display_name]
-> 1158             self.viewer.open(
        self.viewer.open = <bound method ViewerModel.open of Viewer(axes=Axes(visible=False, labels=True, colored=True, dashed=False, arrows=True), camera=Camera(center=(0.0, 0.0, 0.0), zoom=1.0, angles=(0.0, 0.0, 90.0), perspective=0.0, interactive=True), cursor=Cursor(position=(1.0, 1.0), scaled=True, size=1, style=<CursorStyle.STANDARD: 'standard'>), dims=Dims(ndim=2, ndisplay=2, last_used=0, range=((0, 2, 1), (0, 2, 1)), current_step=(0, 0), order=(0, 1), axis_labels=('0', '1')), grid=GridCanvas(stride=1, shape=(-1, -1), enabled=False), layers=[], scale_bar=ScaleBar(visible=False, colored=False, ticks=True, position=<Position.BOTTOM_RIGHT: 'bottom_right'>, font_size=10.0, unit=None), text_overlay=TextOverlay(visible=False, color=<class 'numpy.ndarray'> (4,) float64, font_size=10.0, position=<TextOverlayPosition.TOP_LEFT: 'top_left'>, text=''), overlays=Overlays(interaction_box=InteractionBox(points=None, show=False, show_handle=False, show_vertices=False, selection_box_drag=None, selection_box_final=None, transform_start=<napari.utils.transforms.transforms.Affine object at 0x19832a880>, transform_drag=<napari.utils.transforms.transforms.Affine object at 0x19832a8e0>, transform_final=<napari.utils.transforms.transforms.Affine object at 0x19832a940>, transform=<napari.utils.transforms.transforms.Affine object at 0x19832a9a0>, allow_new_selection=True, selected_vertex=None)), help='', status='Ready', tooltip=Tooltip(visible=False, text=''), theme='dark', title='napari', mouse_move_callbacks=[<function InteractionBoxMouseBindings.initialize_mouse_events.<locals>.mouse_move at 0x1b43161f0>], mouse_drag_callbacks=[<function InteractionBoxMouseBindings.initialize_mouse_events.<locals>.mouse_drag at 0x1b4301670>], mouse_double_click_callbacks=[], mouse_wheel_callbacks=[<function dims_scroll at 0x197778f70>], _persisted_mouse_event={}, _mouse_drag_gen={}, _mouse_wheel_gen={}, keymap={'Shift': <function InteractionBoxMouseBindings.initialize_key_events.<locals>.hold_to_lock_aspect_ratio at 0x1b4301b80>, 'Control-Shift-R': <function InteractionBoxMouseBindings._reset_active_layer_affine at 0x1b428a160>, 'Control-Shift-A': <function InteractionBoxMouseBindings._transform_active_layer at 0x1b428a310>})>
        readerDialog._current_file = '/Users/haase/Downloads/S1_ZF_DIBAC_H2B_Cherry_488_561_tseries_1_MMStack_Default.ome.tif'
        global plugin = undefined
        plugin_name = 'aicsimageio-out-of-memory'
   1159                 readerDialog._current_file,
   1160                 plugin=plugin_name,

~/opt/anaconda3/envs/bio_39/lib/python3.9/site-packages/napari/components/viewer_model.py in open(self=Viewer(axes=Axes(visible=False, labels=True, col...indings._transform_active_layer at 0x1b428a310>}), path='/Users/haase/Downloads/S1_ZF_DIBAC_H2B_Cherry_488_561_tseries_1_MMStack_Default.ome.tif', stack=False, plugin='aicsimageio-out-of-memory', layer_type=None, **kwargs={})
    905             for _path in pbr:
    906                 added.extend(
--> 907                     self._add_layers_with_plugins(
        self._add_layers_with_plugins = <bound method ViewerModel._add_layers_with_plugins of Viewer(axes=Axes(visible=False, labels=True, colored=True, dashed=False, arrows=True), camera=Camera(center=(0.0, 0.0, 0.0), zoom=1.0, angles=(0.0, 0.0, 90.0), perspective=0.0, interactive=True), cursor=Cursor(position=(1.0, 1.0), scaled=True, size=1, style=<CursorStyle.STANDARD: 'standard'>), dims=Dims(ndim=2, ndisplay=2, last_used=0, range=((0, 2, 1), (0, 2, 1)), current_step=(0, 0), order=(0, 1), axis_labels=('0', '1')), grid=GridCanvas(stride=1, shape=(-1, -1), enabled=False), layers=[], scale_bar=ScaleBar(visible=False, colored=False, ticks=True, position=<Position.BOTTOM_RIGHT: 'bottom_right'>, font_size=10.0, unit=None), text_overlay=TextOverlay(visible=False, color=<class 'numpy.ndarray'> (4,) float64, font_size=10.0, position=<TextOverlayPosition.TOP_LEFT: 'top_left'>, text=''), overlays=Overlays(interaction_box=InteractionBox(points=None, show=False, show_handle=False, show_vertices=False, selection_box_drag=None, selection_box_final=None, transform_start=<napari.utils.transforms.transforms.Affine object at 0x19832a880>, transform_drag=<napari.utils.transforms.transforms.Affine object at 0x19832a8e0>, transform_final=<napari.utils.transforms.transforms.Affine object at 0x19832a940>, transform=<napari.utils.transforms.transforms.Affine object at 0x19832a9a0>, allow_new_selection=True, selected_vertex=None)), help='', status='Ready', tooltip=Tooltip(visible=False, text=''), theme='dark', title='napari', mouse_move_callbacks=[<function InteractionBoxMouseBindings.initialize_mouse_events.<locals>.mouse_move at 0x1b43161f0>], mouse_drag_callbacks=[<function InteractionBoxMouseBindings.initialize_mouse_events.<locals>.mouse_drag at 0x1b4301670>], mouse_double_click_callbacks=[], mouse_wheel_callbacks=[<function dims_scroll at 0x197778f70>], _persisted_mouse_event={}, _mouse_drag_gen={}, _mouse_wheel_gen={}, keymap={'Shift': <function InteractionBoxMouseBindings.initialize_key_events.<locals>.hold_to_lock_aspect_ratio at 0x1b4301b80>, 'Control-Shift-R': <function InteractionBoxMouseBindings._reset_active_layer_affine at 0x1b428a160>, 'Control-Shift-A': <function InteractionBoxMouseBindings._transform_active_layer at 0x1b428a310>})>
        _path = '/Users/haase/Downloads/S1_ZF_DIBAC_H2B_Cherry_488_561_tseries_1_MMStack_Default.ome.tif'
        kwargs = {}
        plugin = 'aicsimageio-out-of-memory'
        layer_type = None
    908                         _path, kwargs, plugin=plugin, layer_type=layer_type
    909                     )

~/opt/anaconda3/envs/bio_39/lib/python3.9/site-packages/napari/components/viewer_model.py in _add_layers_with_plugins(self=Viewer(axes=Axes(visible=False, labels=True, col...indings._transform_active_layer at 0x1b428a310>}), path_or_paths='/Users/haase/Downloads/S1_ZF_DIBAC_H2B_Cherry_488_561_tseries_1_MMStack_Default.ome.tif', kwargs={}, plugin='aicsimageio-out-of-memory', layer_type=None)
    950         from ..plugins.io import read_data_with_plugins
    951 
--> 952         layer_data, hookimpl = read_data_with_plugins(
        layer_data = undefined
        hookimpl = undefined
        read_data_with_plugins = <function read_data_with_plugins at 0x1e6fa9280>
        path_or_paths = '/Users/haase/Downloads/S1_ZF_DIBAC_H2B_Cherry_488_561_tseries_1_MMStack_Default.ome.tif'
        plugin = 'aicsimageio-out-of-memory'
    953             path_or_paths, plugin=plugin
    954         )

~/opt/anaconda3/envs/bio_39/lib/python3.9/site-packages/napari/plugins/io.py in read_data_with_plugins(path='/Users/haase/Downloads/S1_ZF_DIBAC_H2B_Cherry_488_561_tseries_1_MMStack_Default.ome.tif', plugin='aicsimageio-out-of-memory')
     94 
     95         hookimpl = hook_caller.get_plugin_implementation(plugin)
---> 96         layer_data = reader(path)
        layer_data = undefined
        reader = functools.partial(<function reader_function at 0x1ca9a9790>, in_memory=False)
        path = '/Users/haase/Downloads/S1_ZF_DIBAC_H2B_Cherry_488_561_tseries_1_MMStack_Default.ome.tif'
     97         # if the reader returns a "null layer" sentinel indicating an empty
     98         # file, return an empty list, otherwise return the result or None

~/opt/anaconda3/envs/bio_39/lib/python3.9/site-packages/napari_aicsimageio/core.py in reader_function(path='/Users/haase/Downloads/S1_ZF_DIBAC_H2B_Cherry_488_561_tseries_1_MMStack_Default.ome.tif', in_memory=False)
    211         return [(None,)]
    212     else:
--> 213         data = _get_full_image_data(img, in_memory=in_memory)
        data = undefined
        global _get_full_image_data = <function _get_full_image_data at 0x1ca9a9f70>
        img = <AICSImage [Reader: OmeTiffReader, Image-is-in-Memory: False]>
        in_memory = False
    214         meta = _get_meta(data, img)
    215         return [(data.data, meta, "image")]

~/opt/anaconda3/envs/bio_39/lib/python3.9/site-packages/napari_aicsimageio/core.py in _get_full_image_data(img=<AICSImage [Reader: OmeTiffReader, Image-is-in-Memory: False]>, in_memory=False)
     36     in_memory: bool,
     37 ) -> xr.DataArray:
---> 38     if DimensionNames.MosaicTile in img.reader.dims.order:
        global DimensionNames.MosaicTile = 'M'
        img.reader.dims.order = undefined
     39         try:
     40             if in_memory:

~/opt/anaconda3/envs/bio_39/lib/python3.9/site-packages/aicsimageio/readers/reader.py in dims(self=<OmeTiffReader [Image-is-in-Memory: False]>)
    509         """
    510         if self._dims is None:
--> 511             self._dims = Dimensions(dims=self.xarray_dask_data.dims, shape=self.shape)
        self._dims = None
        global Dimensions = <class 'aicsimageio.dimensions.Dimensions'>
        global dims = undefined
        self.xarray_dask_data.dims = undefined
        global shape = undefined
        self.shape = undefined
    512 
    513         return self._dims

~/opt/anaconda3/envs/bio_39/lib/python3.9/site-packages/aicsimageio/readers/reader.py in xarray_dask_data(self=<OmeTiffReader [Image-is-in-Memory: False]>)
    336         """
    337         if self._xarray_dask_data is None:
--> 338             self._xarray_dask_data = self._read_delayed()
        self._xarray_dask_data = None
        self._read_delayed = <bound method OmeTiffReader._read_delayed of <OmeTiffReader [Image-is-in-Memory: False]>>
    339 
    340         return self._xarray_dask_data

~/opt/anaconda3/envs/bio_39/lib/python3.9/site-packages/aicsimageio/readers/ome_tiff_reader.py in _read_delayed(self=<OmeTiffReader [Image-is-in-Memory: False]>)
    254             with TiffFile(open_resource) as tiff:
    255                 # Get unprocessed metadata from tags
--> 256                 tiff_tags = self._get_tiff_tags(tiff)
        tiff_tags = undefined
        self._get_tiff_tags = <bound method TiffReader._get_tiff_tags of <OmeTiffReader [Image-is-in-Memory: False]>>
        tiff = <tifffile.TiffFile 'S1_ZF_DIBAC_H2B_…Default.ome.tif'>
    257 
    258                 # Unpack dims and coords from OME

~/opt/anaconda3/envs/bio_39/lib/python3.9/site-packages/aicsimageio/readers/tiff_reader.py in _get_tiff_tags(self=<OmeTiffReader [Image-is-in-Memory: False]>, tiff=<tifffile.TiffFile 'S1_ZF_DIBAC_H2B_…Default.ome.tif'>)
    170 
    171     def _get_tiff_tags(self, tiff: TiffFile) -> TiffTags:
--> 172         unprocessed_tags = tiff.series[self.current_scene_index].pages[0].tags
        unprocessed_tags = undefined
        tiff.series = undefined
        self.current_scene_index.pages.tags = undefined
    173 
    174         # Create dict of tag and value

~/opt/anaconda3/envs/bio_39/lib/python3.9/site-packages/tifffile/tifffile.py in __get__(self=<tifffile.tifffile.lazyattr object>, instance=<tifffile.TiffFile 'S1_ZF_DIBAC_H2B_…Default.ome.tif'>, owner=<class 'tifffile.tifffile.TiffFile'>)
   1074             return self
   1075         try:
-> 1076             value = self.func(instance)
        value = undefined
        self.func = <function TiffFile.series at 0x1bb41cd30>
        instance = <tifffile.TiffFile 'S1_ZF_DIBAC_H2B_…Default.ome.tif'>
   1077         except AttributeError as exc:
   1078             raise RuntimeError(exc)

~/opt/anaconda3/envs/bio_39/lib/python3.9/site-packages/tifffile/tifffile.py in series(self=<tifffile.TiffFile 'S1_ZF_DIBAC_H2B_…Default.ome.tif'>)
   3457             'uniform',
   3458         ):
-> 3459             if getattr(self, 'is_' + name, False):
        global getattr = undefined
        self = <tifffile.TiffFile 'S1_ZF_DIBAC_H2B_…Default.ome.tif'>
        name = 'shaped'
   3460                 series = getattr(self, '_series_' + name)()
   3461                 if not series and name == 'ome' and self.is_imagej:

~/opt/anaconda3/envs/bio_39/lib/python3.9/site-packages/tifffile/tifffile.py in __getattr__(self=<tifffile.TiffFile 'S1_ZF_DIBAC_H2B_…Default.ome.tif'>, name='is_shaped')
   4916             if not self.pages:
   4917                 return False
-> 4918             value = bool(getattr(self.pages[0], name))
        value = undefined
        global bool = undefined
        global getattr = undefined
        self.pages = <tifffile.TiffPages @66837>
        name = 'is_shaped'
   4919             setattr(self, name, value)
   4920             return value

~/opt/anaconda3/envs/bio_39/lib/python3.9/site-packages/tifffile/tifffile.py in __get__(self=<tifffile.tifffile.lazyattr object>, instance=<tifffile.TiffPage 0 @66837>, owner=<class 'tifffile.tifffile.TiffPage'>)
   1074             return self
   1075         try:
-> 1076             value = self.func(instance)
        value = undefined
        self.func = <function TiffPage.is_shaped at 0x1bb422d30>
        instance = <tifffile.TiffPage 0 @66837>
   1077         except AttributeError as exc:
   1078             raise RuntimeError(exc)

~/opt/anaconda3/envs/bio_39/lib/python3.9/site-packages/tifffile/tifffile.py in is_shaped(self=<tifffile.TiffPage 0 @66837>)
   7494         """Return description containing array shape if exists, else None."""
   7495         for description in (self.description, self.description1):
-> 7496             if not description or '"mibi.' in description:
        description = b'\x82\x00\x7f\x00x\x00z\x00}\x00\x80\x00}\x00t\x00{\x00|\x00r\x00\x86\x00\x7f\x00u\x00\x7f\x00|\x00r\x00~\x00\x81\x00z\x00|\x00x\x00n\x00\x84\x00\x7f\x00u\x00y\x00q\x00u\x00\x84\x00x\x00x\x00|\x00|\x00\x7f\x00\x81\x00{\x00w\x00p\x00\x7f\x00\x7f\x00\x81\x00z\x00y\x00v\x00x\x00s\x00q\x00{'
   7497                 return None
   7498             if description[:1] == '{' and '"shape":' in description:

TypeError: a bytes-like object is required, not 'str'

napari-info

napari: 0.4.13
Platform: macOS-12.1-x86_64-i386-64bit
System: MacOS 12.1
Python: 3.9.9 | packaged by conda-forge | (main, Dec 20 2021, 02:41:37) [Clang 11.1.0 ]
Qt: 5.15.2
PyQt5: 5.15.6
NumPy: 1.21.5
SciPy: 1.7.3
Dask: 2021.11.2
VisPy: 0.9.4

OpenGL:
- GL version: 2.1 Metal - 76.3
- MAX_TEXTURE_SIZE: 16384

Screens:
- screen 1: resolution 1800x1169, scale 2.0

Plugins:
- Measurements (Plot profile): 0.2.0
- Measurements (skimage regionprops): 0.2.5
- Search (Plugin): 0.1.2
- Segmentation (split/merge): 0.1.3
- Utilities (skimage regionprops): 0.2.5
- Visualization(B/C): 0.1.5
- aicsimageio-in-memory: 0.4.1
- aicsimageio-out-of-memory: 0.4.1
- clEsperanto: 0.12.0
- console: 0.0.4
- llsz_napari: 0.0.1
- napari-accelerated-pixel-and-object-classification: 0.6.2
- napari-allencell-segmenter: 1.1.3
- napari-clusters-plotter: 0.2.1
- napari-crop: 0.1.3
- napari-curtain: 0.1.0
- napari-folder-browser: 0.1.2
- napari-layer-details-display: 0.1.0
- napari-mouse-controls: 0.1.2
- napari-pyfast-image-processing: 0.0.1
- napari-script-editor: 0.2.5
- napari-segment-blobs-and-things-with-membranes: 0.2.14
- napari-simpleitk-image-processing: 0.1.4
- napari-tabu: 0.1.4
- napari-time-slicer: 0.4.0
- napari-tools-menu: 0.1.8
- napari-workflow-inspector: 0.1.3
- napari-workflow-optimizer: 0.0.1
- napari-workflows: 0.1.1
- natari: 0.2.6
- ome-types: 0.2.9
- scikit-image: 0.4.13
- svg: 0.1.5

cgohlke / tifffile

Opening multi-4GB-OME-TIF file from MicroManager fails #112