A question on warping image

[xiaorugao999]

Hi, @cwmok, I'm sorry to bother you, but I have some questions when I reading the paper and the codes. It will be highly appreciated if you can help me understand the confusion. 1、First is about the input. Why did you use average pooling instead of trilinear interpolation to downsample the moving and fixed image? 2、Second is about the similarity deep supervision in the code. Why did you use the learned deformation field warp the original moving image instead of warping the downsampled moving image in each level? 3、 Third is why did you multiply the output of the final convolution with a self.range_flow instead of directly using it? And why it was set to 0.4?

Thank you again for taking the time out of your busy schedule to help me answer my questions. I am a beginner in this field and have a devout heart to learn.

[cwmok]

Hi @xiaorugao999 ,

Thanks for your interest in our work.

1、First is about the input. Why did you use average pooling instead of trilinear interpolation to downsample the moving and fixed image?

Ans: Both average pooling and trilinear interpolation can be used to resample the input. We prefer average pooling because we would like to remove/smooth the high-frequency components of the input, similar to the traditional Gaussian pyramid used in image registration.

Second is about the similarity deep supervision in the code. Why did you use the learned deformation field warp the original moving image instead of warping the downsampled moving image in each level?

Ans: We actually did learn the deformation field with a multi-level optimization scheme, which we called it progressive training scheme. Besides, in the final stage, your proposed approach will be equal to deep supervision in image segmentation. But we found that the results of your approach are not superior to our approach.

3、 Third is why did you multiply the output of the final convolution with a self.range_flow instead of directly using it? And why it was set to 0.4?

Ans: This is inspired by ICNet. Basically, we want to restrict the maximum deformation potential of the network with the range_flow. Empirically, 0.4 is sufficient for most medical image registration applications.

[xiaorugao999]

Hi, @cwmok , I Really appreciate you taking the time to answer my confusion!!! I have learned a lot from your answers. Thank you so much, But I'm still a little confused about the code, hope you can help me. In the stage of lvl1, i.e. the lowest resolution, the output deformation field output_disp_e0 is the lowest resolution, why it was used to warp the original moving image x which is larger than the deformation field? Meanwhile, I notice that in lvl2, the lvl1_disp_up (upsampling the output_disp_e0) was used to warp x_down, I don't understand why the same deformation field can warp images of different scales. Thank you again for helping me!!!

[cwmok]

HI @xiaorugao999,

The self.transform function uses grid_sample to resample the image. Therefore, the output size of the grid_sample is depending on the size of the defined grid (deformation field + identity), but not depending on the size of input image. Therefore, it can accept images with all scales. You can play around with the grid_sample function.

But there will be a slight difference in using x_down and x as input. When using x_down, you are warping an image downsampled with average pooling. When using x, you are actually warping and downsampling the image using trilinear interpolation. I think both of them are interchangeable, but I haven't tried to switch them.

[xiaorugao999]

Hi, @cwmok, Thank you so much for helping me understand your perfect work！I benefit a lot from your answers, Wish you all the best. yours sincerely xiaoru gao

[xiaorugao999]

Hi, @cwmok, I have another question, why should we apply above operation before computing the grad smooth loss? It seems that VoxelMorph directly computed the smooth loss on the output deformation field.

[cwmok]

Hi @xiaorugao999,

Because F_xy is the normalized velocity field, e.g., [-1, 1] * range_flow. We want to calculate the smoothness loss on the unnormalized velocity field. VoxelMorph also computes the smoothness loss based on the unnormalized displacement field/velocity field.

Of course, calculating the smoothness loss directly on the F_xy also works, but you may need to fine-tune the hyperparameter for the smoothness loss (lambda).

[xiaorugao999]

Hi, @cwmok, Thank you very much. I probably understand. If my network outputs the unnormalized velocity field, do I have to multiply by 2 after upsampling the velocity field before adding it to lvl2's velocity field？

[cwmok]

Yes, you are correct. You need to *2 when using the unnormalized velocity field.