OverflowError in `STalign.LDDMM` and help aligning Visium data

[Nick-Eagles]

Hello,

I'm getting the error OverflowError: cannot convert float infinity to integer when calling the STalign.LDDMM() function. I'll attach the full traceback at the bottom, but I suspect I'm likely giving bad inputs to the function, and so I'll describe a bit more how I'm attempting to use STalign.

I'm attempting to align two Visium H&E samples that were obtained with some overlap in the XY plane but at different depths (partial overlap). I'm loading both images with STalign.normalize(np.array(Image.open(x), dtype = np.float64) / 256) where x is the path to the tissue_hires_image.png file from Spaceranger. This is my first point of confusion, as this tutorial shows loading a Visium H&E a similar way, but I also see from this previous issue (particularly this comment) that perhaps STalign.rasterize should be used instead?

Following along the tutorial, I manually annotate landmarks then find an initial affine transform from them, which is working well. I'm particularly interesting in getting the non-linear transform working though, and invoked Stalign.LDDMM just as in the tutorial:

params = {
    'L':L,
    'T':T,
    'niter': 200,
    'pointsI': pointsI,
    'pointsJ': pointsJ,
    'device': device,
    'sigmaP': 2e-1,
    'sigmaM': 0.18,
    'sigmaB': 0.18,
    'sigmaA': 0.18,
    'diffeo_start': 100,
    'epL': 5e-11,
    'epT': 5e-4,
    'epV': 5e1
}

out = STalign.LDDMM([YI,XI],I,[YJ,XJ],J,**params)

This proceeds for about an hour (using a GPU), ultimately giving the error OverflowError: cannot convert float infinity to integer. Here's the full traceback if it's helpful, though it looks like it's actually occurring in the plotting of the results:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/jhpce/shared/libd/core/stalign/1.0.1/stalign_env/lib/python3.8/site-packages/STalign/STalign.py", line 1306, in LDDMM
    figE.canvas.draw()
  File "/jhpce/shared/libd/core/stalign/1.0.1/stalign_env/lib/python3.8/site-packages/matplotlib/backends/backend_tkagg.py", line 10, in draw
    super().draw()
  File "/jhpce/shared/libd/core/stalign/1.0.1/stalign_env/lib/python3.8/site-packages/matplotlib/backends/backend_agg.py", line 400, in draw
    self.figure.draw(self.renderer)
  File "/jhpce/shared/libd/core/stalign/1.0.1/stalign_env/lib/python3.8/site-packages/matplotlib/artist.py", line 95, in draw_wrapper
    result = draw(artist, renderer, *args, **kwargs)
  File "/jhpce/shared/libd/core/stalign/1.0.1/stalign_env/lib/python3.8/site-packages/matplotlib/artist.py", line 72, in draw_wrapper
    return draw(artist, renderer)
  File "/jhpce/shared/libd/core/stalign/1.0.1/stalign_env/lib/python3.8/site-packages/matplotlib/figure.py", line 3175, in draw
    mimage._draw_list_compositing_images(
  File "/jhpce/shared/libd/core/stalign/1.0.1/stalign_env/lib/python3.8/site-packages/matplotlib/image.py", line 131, in _draw_list_compositing_images
    a.draw(renderer)
  File "/jhpce/shared/libd/core/stalign/1.0.1/stalign_env/lib/python3.8/site-packages/matplotlib/artist.py", line 72, in draw_wrapper
    return draw(artist, renderer)
  File "/jhpce/shared/libd/core/stalign/1.0.1/stalign_env/lib/python3.8/site-packages/matplotlib/axes/_base.py", line 3064, in draw
    mimage._draw_list_compositing_images(
  File "/jhpce/shared/libd/core/stalign/1.0.1/stalign_env/lib/python3.8/site-packages/matplotlib/image.py", line 131, in _draw_list_compositing_images
    a.draw(renderer)
  File "/jhpce/shared/libd/core/stalign/1.0.1/stalign_env/lib/python3.8/site-packages/matplotlib/artist.py", line 72, in draw_wrapper
    return draw(artist, renderer)
  File "/jhpce/shared/libd/core/stalign/1.0.1/stalign_env/lib/python3.8/site-packages/matplotlib/axis.py", line 1388, in draw
    ticks_to_draw = self._update_ticks()
  File "/jhpce/shared/libd/core/stalign/1.0.1/stalign_env/lib/python3.8/site-packages/matplotlib/axis.py", line 1275, in _update_ticks
    major_labels = self.major.formatter.format_ticks(major_locs)
  File "/jhpce/shared/libd/core/stalign/1.0.1/stalign_env/lib/python3.8/site-packages/matplotlib/ticker.py", line 218, in format_ticks
    return [self(value, i) for i, value in enumerate(values)]
  File "/jhpce/shared/libd/core/stalign/1.0.1/stalign_env/lib/python3.8/site-packages/matplotlib/ticker.py", line 218, in <listcomp>
    return [self(value, i) for i, value in enumerate(values)]
  File "/jhpce/shared/libd/core/stalign/1.0.1/stalign_env/lib/python3.8/site-packages/matplotlib/ticker.py", line 1088, in __call__
    is_x_decade = _is_close_to_int(fx)
  File "/jhpce/shared/libd/core/stalign/1.0.1/stalign_env/lib/python3.8/site-packages/matplotlib/ticker.py", line 2248, in _is_close_to_int
    return math.isclose(x, round(x))
OverflowError: cannot convert float infinity to integer

Thanks so much in advance! This seems like a powerful tool and I'm excited to get it working.

Best, -Nick

[Nick-Eagles]

Update: I tried the same code but with lower resolution images, and out = STalign.LDDMM([YI,XI],I,[YJ,XJ],J,**params) runs without the error, but out['A'] is mostly nan:

tensor([[nan, nan, nan],
        [nan, nan, nan],
        [0., 0., 1.]], device='cuda:0', dtype=torch.float64)

[JEFworks]

Dear Nick,

This tutorial for aligning Visium datasets may be useful to you: https://jef.works/blog/2023/11/05/aligning-visium-using-STalign-with-reticulate/ In general, for aligning two Visium datasets, I would recommend aligning the Cytassist images and then applying the transformation onto the spots to move them into the aligned coordinate space (rather than rasterizing spots because spots are uniformly gridded and contain no cell density information in their positions)

This tutorial also described some matrix dimensions and parameters to be careful about: https://jef.works/blog/2024/04/11/stalign-at-single-cell-resolution/

Hope this helps, Jean