Non Rigid Registration Error

[Tejussurendran]

Hello,

I have recently learned about valis for image registration and I tried to use it on 2 images with different stains to register the AT8 image to the reference H&E image. However when It completes the non-rigid registration process it proceeds to throw and error that I am not able to resolve. I was wondering if there are any suggestions to resolve this issue

Thank you! I have attached a copy of the error message below.

[cdgatenbee]

Hi @Tejussurendran, Sorry to see you're having this issue. It looks like the registration was successful, but that the error was thrown when trying to create some visualizations to asses the quality of the registration. Specifically, it's when valis is warping a triangular mesh to draw overtop the non-rigid registered image so that you can see where and how extreme the non-rigid deformations are.

The problem seems to be with the interpolation package not liking the format of the numbers being passed to UCGrid, which is used to warp the mesh vertices. It looks like the interpolation package was updated a few days ago so I'll take a look to see what changed and then update valis with the fix. I'll let you know when it's been patched.

Best, -Chandler

[Tejussurendran]

Thank you so much for your help!

[cdgatenbee]

Hi @Tejussurendran , I've updated the repo and pypi package to fix this error (vesion 1.0.0rc10). If you're still having issues please let me know.

Best, -Chandler

[Tejussurendran]

Hi @cdgatenbee,

Thank you so much for your help!

I reinstalled the package and I believe it is working now.

However I did have a question regarding how to save the newly registered images.

I saw the function warp_and_save_slide() however it saves the images as a tiff file, is it possible to save them as SVS files by any chance?

Thank you for your help!

Tejus

[cdgatenbee]

Hi @Tejussurendran, Unfortunately, it's only possible to save the the full resolution slides in the ome.tiff format, and I'm not really sure if there is software that can save as svs. However, ome.tiff has features similar to svs, in that they are tiled, pyramidal (multi-resolution), and have metadata about physical units, channel names, etc... Like svs, ome.tiff can also be opened using several different packages (via bio-formats, tifffile, pyvips, valis, etc...) and/software (QuPath, ImageJ, HALO, etc), so hopefully you'll still be able to use the saved slides in your current pipeline. If you don't mind my asking, is there are particular reason you would need svs?

Best, -Chandler

[Tejussurendran]

Hi @cdgatenbee,

Thank you for your response. The reason I was asking about SVS files is that I am using the image registration on one particular stain, however I also have a set a batch of H&E images that are already svs files, and I wasnt sure if the codebase I was working with would be able to access ome.tiff filess. However, I dont think it should be an issue as it appears OpenSlide can open the ome.tiff files. However, I was a little confused as to why the ome.tiff file sizes were significantly larger than the original svs files. As the registered Tiff file was 6.8GB while the original image was somewhere around 680MB.

Thank you for your help!

[Tejussurendran]

@cdgatenbee

I am relatively new to medical imaging, and one of the things I was trying to understand is how come there is only 1 level when looking at the .ome.tiff files, compared to an svs file it appears there are 3 levels.

The issue I'm having right now is that these images are too big to segment in their native resolution and it appears that there are no other levels that I can use to downsample the image.

[cdgatenbee]

Hi @Tejussurendran , Wow, that is a pretty big difference in file size. The default compression method is LZW, but maybe a different compression method would help? Valis uses pyvips to save the images, and so the different compression methods can found at https://libvips.github.io/pyvips/enums.html#pyvips.enums.ForeignTiffCompression. Just set the "compression" argument to whichever method you would like to use when calling warp_and_save_slide.

Here is how you would store the warped image as a numpy array:

`

from valis import warp_tools

# Get the Slide object associated with the slide you'd like to warp
slide_obj = registrar.get_slide(slide_filename)

# Check the image dimensions to determine which resolution you would like to warp.
slide_dimensions = slide_obj.slide_dimensions_wh
image_level = 2
# Warp the image and get a pyvips.Image back. Shouldn't matter how large the image is
# Will warp an image with width and height = slide_obj.slide_dimensions_wh[image_level]
warped_slide = slide_obj.warp_slide(image_level)  

# Convert to numpy array. Image will be opened in memory, so need to be careful if the image is really large
numpy_warped_slide = warp_tools.vips2numpy(warped_slide)

`

If you just want part of the image you can use the extract the region of interest using warped_slide.extract_area(x, y, w, h) before converting to a numpy array

I realized I don't have a simple example like this in the README, so I'll be sure to add one.

[cdgatenbee]

@Tejussurendran, I just saw your new question. The saved ome.tiff should have several resolutions, so let me know if the above method works when you're trying to get a lower resolution image. When you see that there is only 1 level, which software are you using?

Also, I'm not sure what sort of segmentation you're doing, but if it's something like cell segmentation you could process the image in smaller tiles using warped_slide.extract_area(x, y, w, h), where each xywh is for a particular tile.

[Tejussurendran]

Hi @cdgatenbee,

Thank you for your response!

I will definitely go ahead and try to use the new method you had suggested.

However to answer your question, I am using OpenSlide to read the ome.tiff(which i believe is at level 0, and from my understanding that should mean native resolution) file and compare it to the original SVS file.

When I go to get the level_dimensions, and ((65735, 45957), (16433, 11489), (4108, 2872), (2054, 1436)) ((67727, 45960),). The first being the svs file and the latter being the ome.tiff file.

Ideally, I would like to have an image close to the original resolution, however have multiple layers afterwards, so I could potentially create a thumbnail if necessary.

I have attached the code I used below to create this ome.tiff file, just to make sure that I didn't make any errors.

import time import os from valis import registration

slide_src_dir = "/SeaExpCIFS/Restricted/Registration_test" results_dst_dir = "/SeaExpCIFS/Restricted/Registration_test_output"

Create a Valis object and use it to register the slides in slide_src_dir

start = time.time() reference_slide = "/SeaExpCIFS/Restricted/Registration_test/4313.svs" registrar = registration.Valis(slide_src_dir, results_dst_dir,reference_img_f=reference_slide,align_to_reference=True) rigid_registrar, non_rigid_registrar, error_df = registrar.register() stop = time.time() elapsed = stop - start print(f"regisration time is {elapsed/60} minutes")

Check results in registered_slide_dst_dir. If they look good, export the registered slides

registered_slide_dst_dir = os.path.join("/SeaExpCIFS/Restricted/expected_results/registered_slides", registrar.name) start = time.time() slide_obj = registrar.get_slide('/SeaExpCIFS/Restricted/Registration_test/4605.svs') slide_obj.warp_and_save_slide("/SeaExpCIFS/Restricted/expected_results/4605_registered.ome.tiff", level=0)

registrar.warp_and_save_slides(registered_slide_dst_dir)

stop = time.time() elapsed = stop - start print(f"saving {registrar.size} slides took {elapsed/60} minutes")

Shutdown the JVM

registration.kill_jvm()

[cdgatenbee]

Everything looks good @Tejussurendran :)

I took a look at the OpenSlide website, and it doesn't appear that it officially supports ome.tiff images. I'm guessing maybe what is happening is that OpenSlide is treating the image as a regular tiff, and so it doesn't "see" the pyramid levels (ome-tiffs have a pretty different structure). Just a guess, but do please let me know if you're not able to access the other resolutions using valis or something like QuPath.

Regarding the image dimensions, the registered image probably won't have the same shape as the unregistered one. That's because the registered image shape has to be large enough accommodate the sizes of all of the images being aligned. However, this is usually pretty large (especially when there are rotations), and there is a lot of empty space. So valis defaults to cropping the registered images such that it contains only the areas where all of the warped images overlap. However, it is also possible to specify that the registered images be cropped so that they fit within a reference image. You can specify this image when you create the Valis object, by setting reference_img_f to be the name of your reference slide (i.e. the one you want the others to align to and which also won't get warped). If you don't set one, valis will pick it, and you can see which it is by looking at the reference_img_f attribute of your Valis object. Anyway, if you'd like to crop the registered images to have the same shape as the reference image, set the "crop" to registration.CROP_REF when you call warp_slide.