Collect example data

[K-Meech]

Definition of Done

Collect example data for the different episodes e.g.

• Small (fast to download) images to show different image types - 2D / 3D / multichannel / RGB / time series.

• A clean dataset for a cell counting exercise

• A messier/more realistic dataset for cell counting

• A larger dataset for the long-form example on the final day e.g. histopathology data on omero

• Testing is not a part of Definition of Done of this task (data will be tested along with lessons).

Smaller datasets could be placed directly in the repo or uploaded to Figshare / Zenodo for download e.g. the data carpentry image processing lesson stores example data in figshare

[K-Meech]

I looked through the example data that comes with napari. These images can easily be loaded via the user interface using File > Open Sample... They are the same example images as scikit-image's data

The most relevant ones are:

• Human mitosis (2D grayscale)
• Cells (3D + 2Ch) - 3D fluorescence microscopy data with 2 channels
• Kidney (3D + 3Ch) - 3D fluorescence microscopy data with 3 channels
• Skin (RGB) - 2D hematoxylin + eosin stained slide
• Immunohistochemistry (RGB) - 2D IHC staining with hematoxylin counterstaining

Some quick screenshots:

[davecash75]

HI @K-Meech what formats are these in? Or are they directly loaded into napari using the skimage commands?

[K-Meech]

@davecash75 These are loaded directly into napari via the user interface (which I believe uses skimage commands in the background) - so they just appear as numpy arrays, rather than images in specific file formats. These are nice for showing imaging basics, but we'd need other examples to show real metadata + file formats. We could consider looking on BioImage Archive if we needed more example data from real experiments: https://www.ebi.ac.uk/bioimage-archive/

[K-Meech]

QuPath has some small example pathology images available for download: https://qupath.readthedocs.io/en/stable/docs/intro/acknowledgements.html

[davecash75]

What about Omero's sample data: https://downloads.openmicroscopy.org/images/

[p-j-smith]

Here's another list of potential resources: https://github.com/AllenCellModeling/aicsimageio/discussions/544#discussioncomment-7568092

[MonikaSvata]

Waiting for Nick to send example data.

[davecash75]

Nic has sent some over and @K-Meech has access.

[MonikaSvata]

Dave to get back to Nic requesting more info about the data and see if he has smaller data sets.

[davecash75]

One thing that is a bit unfortunate @K-Meech @drmatthews @dpshelio is that when I tried opening up the Zeiss CZI file that Nic provided (about 400MB), I get some GL_TEXTURE warnings saying the data size is bigger than the GL Texture mapping limits on my laptop (16484) and there is some downsampling performed. However, Fiji opens this file up with no problem and lots more extensive support and options on how to import this data. On the other hand, I didn't know what half of these options mean - I would expect I would need an episode on what that one dialog box itself means. Much less how to navigate the images in Fiji

[davecash75]

Actually the confocal file 63x zstack LSMplus.czi opens up in my Napari and looks quite nice, save for the metadata, which doesn't show up in the OME plugin, and on the terminal it spits out the following errors

SLF4J: No SLF4J providers were found.
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See https://www.slf4j.org/codes.html#noProviders for further details.
/Users/davecash/anaconda3/envs/napari-env/lib/python3.9/site-packages/ome_types/_mixins/_base_type.py:100: UserWarning: Casting invalid DichroicID 'Beamsplitter.2412-349' to 'Dichroic:0'!
/Users/davecash/anaconda3/envs/napari-env/lib/python3.9/site-packages/ome_types/_mixins/_base_type.py:100: UserWarning: Casting invalid DichroicID 'Beamsplitter.2412-356' to 'Dichroic:1'!
/Users/davecash/anaconda3/envs/napari-env/lib/python3.9/site-packages/ome_types/_mixins/_base_type.py:100: UserWarning: Casting invalid DichroicID 'SecondaryBeamsplitter.2415-596' to 'Dichroic:2'!
/Users/davecash/anaconda3/envs/napari-env/lib/python3.9/site-packages/xsdata/formats/converter.py:85: ConverterWarning: Failed to convert value `` to one of (<class 'float'>,)!
/Users/davecash/anaconda3/envs/napari-env/lib/python3.9/site-packages/ome_types/_mixins/_base_type.py:100: UserWarning: Casting invalid LightSourceID 'MTBLKM980LaserLine488' to 'LightSource:0'!
/Users/davecash/anaconda3/envs/napari-env/lib/python3.9/site-packages/ome_types/_mixins/_base_type.py:100: UserWarning: Casting invalid DichroicID 'Beamsplitter.2412-349' to 'Dichroic:3'!
/Users/davecash/anaconda3/envs/napari-env/lib/python3.9/site-packages/ome_types/_mixins/_base_type.py:100: UserWarning: Casting invalid DichroicID 'Beamsplitter.2412-356' to 'Dichroic:4'!
/Users/davecash/anaconda3/envs/napari-env/lib/python3.9/site-packages/ome_types/_mixins/_base_type.py:100: UserWarning: Casting invalid DichroicID 'SecondaryBeamsplitter.2415-596' to 'Dichroic:5'!
/Users/davecash/anaconda3/envs/napari-env/lib/python3.9/site-packages/xsdata/formats/converter.py:85: ConverterWarning: Failed to convert value `` to one of (<class 'float'>,)!
/Users/davecash/anaconda3/envs/napari-env/lib/python3.9/site-packages/ome_types/_mixins/_base_type.py:100: UserWarning: Casting invalid LightSourceID 'MTBLKM980LaserLine488' to 'LightSource:1'!

[K-Meech]

@davecash75 - yes Fiji currently has much better bioformats support than napari + handles a much wider range of file formats. I've made sure to mention Fiji in the filetypes/metadata episode, as it's often the best option to open these proprietary file formats!

According to the napari-aicsimageio docs, it has pretty strict limits on which images it will load at full size. Only those less than 4GB total size and less than 30% of total memory. This is a shame if you're using a workstation that could handle much larger images directly!

[MonikaSvata]

Decision made to used one of the Napari images.

@K-Meech to add notes about limitations - after this it can be closed.

[K-Meech]

I added a note about the file size limitations of napari-aicsimageio to the 'filetypes and metadata' episode. I'll close this issue now.