search

cwmok / LapIRN

Large Deformation Diffeomorphic Image Registration with Laplacian Pyramid Networks
MIT License
118 stars 20 forks source link

DVF composition or addition #19

Closed xiaogengen-007 closed 2 years ago

xiaogengen-007 commented 2 years ago

Hi, @cwmok, Thank you so much for the great work and publishing the code!

I have a question about the composition of the deformation vector field (DVF). I am doing non-diffeomorphic registration using Train_LapIRN_disp.py. I notice you use direct addition when combining the current level DVF with previous level DVF: compose_field_e0_lvl1 = lvl1_disp_up + output_disp_e0_v compose_field_e0_lvl1 = output_disp_e0_v+lvl2_disp_up

If I understand correctly, DVF composition tends to be better than addition when combining DVFs. (For example, see Section 6.4 in the CycleMorph paper(https://www.sciencedirect.com/science/article/pii/S1361841521000827). I wonder if there's any particular reason for using addition instead of composition here.

I have some naive guesses, but I don't know if it is correct: I think the addition makes sense to some degree because you are not combining the multilevel DVFs in the end. As the DVF addition is performed within the network, the network might be able to learn to predict DVFs that are "addable".

Thank you in advance!

cwmok commented 2 years ago

Hi @xiaogengen-007,

Your initial guess is almost correct. We are not combining the multilevel DVFs in the end. Instead, LapIRN first estimates a coarse solution in the first level and aims to improve the "coarse" solution in the subsequent levels. It is similar to the Laplacian image pyramid or deep Laplacian network, which saves/predicts the difference image of the blurred versions between each level.

Having said that, both composition and addition will work with LapIRN since we jointly optimized all the levels at the final stage. We prefer addition over composition for two main reasons: 1) addition is more computational effective and 2) the solutions using addition tend to be smoother than those using composition in our experiments (Brain MR registration using OASIS). We speculate the reason is that the boundary of the solutions using composition is not well-defined, e.g., zero-padding/reflection.

Yet, composition and addition may not be interchangeable if you are using a conventional multi-level optimization scheme like [mlVIRNET](mlVIRNET: Multilevel Variational Image Registration Network). Since these methods optimize multiple networks separately and the coarse solutions are not used as the input of the network, they should use composition operator to combine solutions from different levels.

There are some properties of the composition. For example, two diffeomorphic deformation fields combined with the composition operator will result in a diffeomorphic solution, but it is not the case for addition.

xiaogengen-007 commented 2 years ago

Thank you so much for this super clear and detailed answer! It makes perfect sense. I really appreciated it!