Stitching multiple fields of views

[dganguli]

Currently, Starfish processes one field of view at a time. At some point, these processed (and potentially unprocessed) images/results need to be stitched together for visualization. This can either be done implicitly, e.g., for each FOV we record an offset and position in an overall grid with necessary overlap information, or explicitly, e.g., the former information is used to create one large image / table representing results.

What are the right algorithms for stitching, handling boundary artifacts, and de-duping tabular results data?

[ambrosejcarr]

Notes for self about some available (implemented) methods: https://github.com/linnarsson-lab/pysmFISH/tree/master/pysmFISH/stitching_package https://en.wikipedia.org/wiki/Image_stitching http://scikit-image.org/docs/dev/api/skimage.measure.html http://scikit-image.org/docs/dev/api/skimage.measure.html#skimage.measure.ransac https://github.com/freeman-lab/regional/blob/master/regional/regional.py#L73 https://ieeexplore.ieee.org/document/6396024/?reload=true

Potentially naive thoughts:

• We could register the images using existing approaches (RANSAC, SIFT), or we could use our extracted dots as features, and register on them.
  • Merging segmentation regions (which we have already transformed into polygons) should be relatively easy. We would need to embed regional.One objects into global coordinate space, shift them according to registration changes, and then use the already-implemented merge method.

[ambrosejcarr]

Also Render: https://github.com/saalfeldlab/render