search

BailiangJ / spine-ct-mr-registration

Weakly-supervised Biomechanically-constrained CT/MRI Registration of the Spine
MIT License
22 stars 3 forks source link

Should the rigidity loss (PC and OC) be applied on the fixed image mask or moving image mask? #1

Closed xiaogengen-007 closed 1 year ago

xiaogengen-007 commented 1 year ago

Hi BailiangJ,

Thank you for your great MICCAI paper and for making the code public!

I am also working on DL registration and I have a question about the rigid mask. If I understand correctly, the PC and OC loss in your paper was applied within the bony mask from the moving image:

“Compared to previous supervised DLIR methods, our approach only requires the moving label (MC 1).”

And I have read the paper "A rigidity penalty term for nonrigid registration" by Staring et al, where they also did it on the moving image:

"The rigidity coefficient image only has to be defined on the moving image. The moving image deforms to the fixed image, so specific parts of the moving image should transform in a rigid fashion. Therefore, the regions that correspond to rigid structures have to be defined on the moving image."

However, when I read the voxelmorph paper and the Elastix manual, I personally think the rigidity loss should be applied on the mask from the fixed image. I have attached the evidence below, but in short the key point is that the deformation field is defined in the fixed image domain. If the deformation field is defined in the fixed image domain, I think the rigidity loss should also be defined on the mask from the fixed image (instead of the moving).

That being said, I do see in both Staring's and your paper, the rigidity loss is defined on the moving mask. I am really confused and stuck at this point and I don't know whether or where I made a mistake. Could you please give me any insights on this issue? Thank you so much in advance.

Evidence from voxelmorph and Elastix manual:

  1. In this GitHub issue from voxelmorph, the author mentioned:

    "When warping image moving to image fixed, the deformation field is in the space of the fixed image -- it tells you how to move the moving image to get to each grid location in the fixed image. For example, to get to location (123, 143) into the fixed space, it might tell you to grab (2.2, 1.3) voxels from the moving image. The reason this is a bit "backwards" is because moving pixels in this way enables us to interpolate from the moving image -- that is, we can get the "intensity value" at location (125.2, 144.3) from the moving image by bilinear interpolation, and move it to (123, 143)."

  2. In the Elastix manual:

    image image

BailiangJ commented 1 year ago

Hi @xiaogengen-007 ,

yes, you are correct, the deformation field output by Voxelmorph network is mapping from the fixed image domain to the moving image domain. Therefore, in the OC and PC loss, we are using the warped moving label (y_source_oh) to mask the output deformation field, and the warped moving label is in the fixed image domain.

Additionally, since we are using the diffeomorphic version of Voxelmorph, you can also apply PC and OC loss with moving label and inverse deformation (mapping from moving image domain to fixed image domain).

Let me know if these are clear to solve the problem:)

xiaogengen-007 commented 1 year ago

Hi @BailiangJ,

Thank you so much for the clarification! It solved my problem totally!

I agree that it might be better to use the warped moving label instead of the fixed label, considering that the registration might not be perfect (i.e. we can't guarantee a perfect match between fixed label and warped moving label). However, I did some quick experiments on our CT registration problem and found that it didn't make much of a difference whether I use the fixed label or the warped moving label.

Anyway, thank you so much for your prompt and detailed reply!