Odd transformed image following the bspline example

[asmagen]

I was trying to apply a pyramidal bspline registration as described in the diffeomorphic_bspline_2d.py script here, but after applying the transformation to the RGB channels I got the following odd result:

Any idea why? Thanks

[jlevy44]

I am getting this same issue as well!

[RobinSandkuehler]

Hi,

do you get the same error if you apply the transformation only on one channel?

[jlevy44]

I think it's coming from https://github.com/airlab-unibas/airlab/blob/master/examples/kernel_registration_2d.py#L64

@sumanthratna

[jlevy44]

I had acquired the displacement field from application to the grayscale, then independently applied it to each color channel, so it should have been fine, I think the issue may be with the interpolation as we move to alignment of larger images within the pyramid image scheme, but I am unsure.

[sumanthratna]

It looks like this is happening with airlab.transformation.pairwise.WendlandKernelTransformation too.

[RobinSandkuehler]

What size are your images. Do you try to change the kernel size?

[sumanthratna]

What size are your images.

My images are 2330  px by  2586 px and 2357 px  by  2083 px.

Do you try to change the kernel size?

How would I do this? I'm currently setting sigma to (1, 1). Setting cp_scale to 1 doesn't make much of a difference. https://github.com/airlab-unibas/airlab/blob/80c9d487c012892c395d63c6d937a67303c321d1/airlab/transformation/pairwise.py#L655-L669

[RobinSandkuehler]

I would recommend not to set the sigma of the kernel to 1. With images of your size you should also increase the number of pyramid levels. You can try to set the sigma to 50 and try again. Please let me know if this works. Could you share the images? Then I can try the registration.

[sumanthratna]

Here're the input images I'm currently working with:

Setting sigma to 1 results in this output:

Setting sigma to (50, 50) results in:

Setting sigma to 1 minimizes the magnitude of this unexpected effect.

[RobinSandkuehler]

Thanks for sharing. I will try to register them and let you know as soon as possible.

[RobinSandkuehler]

Hi, I tried to register your images and I think I found a solution that may help you.

Here are the things I changed in the 2d Bspline example:

read the images fixed_image = al.read_image_as_tensor("image_1", dtype=dtype, device=device) moving_image =al.read_image_as_tensor("image_2", dtype=dtype, device=device)

normalise the images fixed_image, moving_image = al.utils.normalize_images(fixed_image, moving_image)

bring them on a joint domain fixed_image, f_mask, moving_image, m_mask, cm_displacement = al.get_joint_domain_images(fixed_image, moving_image,default_value=1, cm_alignment=False, compute_masks=False)

create an image pyramide size/8, size/4, size/2, size/1 fixed_image_pyramid = al.create_image_pyramid(fixed_image, [[8, 8], [4, 4], [2, 2]]) moving_image_pyramid = al.create_image_pyramid(moving_image, [[8, 8], [4, 4], [2, 2]])

then I used these parameters for the registration regularisation_weight = [100, 100, 10000, 100000] number_of_iterations = [500, 250, 100, 10] sigma = [[50, 50], [50, 50], [75, 75], [100, 100]]

On of the most important change is in the image similarity function. I use the normalised Cross Correlation (NCC) instead of the Mean square error (MSE). This is important as I assume that you images with different stainings (please correct me if I am wrong).

image_loss = al.loss.pairwise.NCC(fix_im_level, mov_im_level)

I hope this can help you for your registration problem. Please let me know if you have further questions.

[jlevy44]

Thanks! Seems like regularization is key here.

Does get_joint_domain_images alter the final application of the displacement field when warping color channels independently?

[RobinSandkuehler]

Yes regularisation and the loss. Actually you have to trim the displacement to the original size since get_joint_domain_images changes the image size.

[sumanthratna]

It looks like setting the learning rate to a very small value seems to fix this in both B-spline and Wendland.

[asmagen]

@sumanthratna How do I adjust that learning rate or regularization? Did you revise that in PathFlow-MixMatch or is it pending revision?

[asmagen]

@RobinSandkuehler Regarding your comment about images with different marker stainings, I was thinking it might be best to register the hematoxylin grayscale image generated by color deconvolution (separate_stains in scikit learn). Would that be a good approach? Would I then use MSE instead of NCC?

[sumanthratna]

@sumanthratna How do I adjust that learning rate or regularization? Did you revise that in PathFlow-MixMatch or is it pending revision?

@asmagen Sorry for the late reply. You should already be able to pass those in via the command line. We should have a fix coming soon for this issue.

---

A quick heads-up for anyone else seeing this issue—along with changing the regularization weights and loss function as @RobinSandkuehler suggested, try changing diffeomorphic to False when instantiating the transformation.