Realtime Stitch

[alexlsa]

Hi,

Thanks for the project

Is it possible to realtime-stitch for brightfield microscopy images 2D ?

[m-albert]

@alexlsa You can definitely stitch brightfield microscopy images. Have a look at this notebook for a starting point on how to register and fuse 2D image tiles.

If by "realtime" you refer to the speed, you'd need to test this for your configuration (would mostly depend on the number of image tiles and their sizes). If you refer to automatically processing during the acquisition, you'd need some additional logic to call the multiview-stitcher functions during acquisition.

[alexlsa]

First of all thank you @m-albert

I have looked your shared notebook and I achieved to stitch below grayscale images, normally they are color image but I converted them to grayscale to stitch them. To stitch color images, I have to modify some things

By "realtime" I mean automatically processing during the acquisition because my images is 5mp resolution and stitching after acqusition takes much time. If I stitch them during the acquisition, this makes me save much time.

I'm trying to understand the code right now, Do you think we need to make too many changes for the live stitch process ?

image12_29: image12_30:

Stitched:

All in one:

[m-albert]

This doesn't look too bad!

I have looked your shared notebook and I achieved to stitch below grayscale images, normally they are color image but I converted them to grayscale to stitch them. To stitch color images, I have to modify some things

Have a look at this code snippet that shows how to load data from numpy arrays that can also contain a channel dimension. In registration.register you then use the argument registration_channel_index to set the channel to use for registration. The code snippet is for 3d data but you can simply take away the z dimension to adapt it for 2d.

import dask.array as da

from multiview_stitcher import spatial_image_utils as si_utils
from multiview_stitcher import msi_utils, vis_utils

tile_translations = [
    {'z': 2.5, 'y': -30, 'x': 0},
    {'z': 2.5, 'y': 30, 'x': 10},
    {'z': 2.5, 'y': 30, 'x': 50},
]

tile_scales = [{'z': 2, 'y': 0.5, 'x': 0.5}] * len(tile_translations)

channels = ['DAPI', 'GFP']

msims = []
for tile_translation, tile_scale in zip(tile_translations, tile_scales):
    print(tile_translation, tile_scale)

    sim = si_utils.get_sim_from_array(
        da.random.randint(0, 100, (2, 10, 100, 100)),
        dims=['c', 'z', 'y', 'x'],
        scale=tile_scale,
        translation=tile_translation,
        transform_key='stage_metadata',
        c_coords=channels,
    )

    msim = msi_utils.get_msim_from_sim(sim, scale_factors=[])
    msims.append(msim)

vis_utils.plot_positions(msims, transform_key='stage_metadata', use_positional_colors=False)

I'm trying to understand the code right now, Do you think we need to make too many changes for the live stitch process ?

As far as I understand it, you'd mostly need a script that looks out for new files appearing and stitching them.

[alexlsa]

Yes I think it looks good too, I also have a mistake because I set overlap 0.1 but my images has 0.12 overlap. but sitll it looks good.

I will try your code to stitch color images also, thank you

As far as I understand it, you'd mostly need a script that looks out for new files appearing and stitching them.

Actually yes if there is a new file or array, then stitch them. That way I can stitch images when performing acqusition

If there is a task I can do, I will be happy to do it, but I am not fully familiar with your algorithm.

Thank you

[m-albert]

Yes I think it looks good too, I also have a mistake because I set overlap 0.1 but my images has 0.12 overlap. but sitll it looks good.

The registration step should take care of correcting this.

Actually yes if there is a new file or array, then stitch them. That way I can stitch images when performing acqusition If there is a task I can do, I will be happy to do it, but I am not fully familiar with your algorithm.

To be clear, this package provides functions to stitch (=register and fuse) your data, but it doesn't provide functionality for calling these functions during acquisition. To do that you could e.g.

• write a script with an infinite loop that lists all files of a folder and stitches the files that haven't been stitched yet or
• use a tool like watchdog for this.