Registration Tutorial for DVF prediction

[kate-sann5100]

Is your feature request related to a problem? Please describe. There are three distinct registration networks (UNet, LocalNet, GlobalNet) and two types of prediction (dvf and ddf) supported by MONAI. The paired_lung_ct demos combines LocalNet and ddf prediction. It would be great if we could provide some more tutorials with different network (GlobalNet) and prediction types (dvf).

Describe the solution you'd like Add new tutorials.

Describe alternatives you've considered N/A

Additional context N/A

[kate-sann5100]

Should we

1. use GlobalNet + dvf on the paired_lung_ct task again OR
2. use other data and tasks? @mathpluscode @YipengHu

[YipengHu]

Should we

1. use GlobalNet + dvf on the paired_lung_ct task again OR
2. use other data and tasks? @mathpluscode @YipengHu

LocalNet/UNet + DVF makes more sense. I'd suggestion the two tutorials be: 1- LocalNet/UNet+DVF, unpaired, no label supervision - using single or mixed inhale/exhale phases; 2- LocalNet, paired, with label supervision - registering inhale (moving) to exhale (fixed) phases. if time, 3 - Change one of the above to GlobalNet (DVF does not have effect if affine here)

[kate-sann5100]

@YipengHu That sounds very feasible. Btw I am curious what is the pro and con of DVF vs DDF and their suitable use cases?

[YipengHu]

Roughly speaking, the velocity vectors will be integrated to displacement vectors. The resulting DDF has a set of properties that is useful to model some biological tissue deformation or correspondence finding, such as invertibility, one-to-one mapping, volume preserving - differmorphism.

While Predicting DDF is less constrained, more efficient and more suitable for modelling transformation that does not fit the above description - obviously we also add explicit deformation regularisation term, so things a re a bit more complicated.

@Vercauteren, Tommailto:tom.vercauteren@kcl.ac.uk corrects me please.

Yipeng

---

From: Yiwen Li notifications@github.com Sent: Friday, March 5, 2021 6:37:45 PM To: Project-MONAI/tutorials tutorials@noreply.github.com Cc: Hu, Yipeng yipeng.hu@ucl.ac.uk; Mention mention@noreply.github.com Subject: Re: [Project-MONAI/tutorials] Registration Tutorial for DVF prediction (#134)

That sounds very feasible. Btw I am curious what is the pro and con of DVF vs DDF and their suitable use cases?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/Project-MONAI/tutorials/issues/134#issuecomment-791605963, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AATX37ND5H7Z5K4CWQTONYLTCEQHTANCNFSM4YVON6VA.

[kate-sann5100]

@YipengHu That answers my question perfectly. So does that mean in the lung ct case, dvf has no obvious advantage over ddf as we don't have specific constraints?

[YipengHu]

@YipengHu That answers my question perfectly. So does that mean in the lung ct case, dvf has no obvious advantage over ddf as we don't have specific constraints?

Now this is still an open research question as far as I'm aware. For example, between rib cage and lung tissue, there is clear sliding motion which, in theory, should not be modelled as diffeomorphic transformation, plus the airway expansion etc, but dvf may still perform better locally especially when the corresponding features are lacking.

[kate-sann5100]

@YipengHu I think this information is enough for me to write the tutorials for now. That's super helpful. Thank you.

[tvercaut]

Not much more to add from @YipengHu's answer. Just one clarification: Even with stationary velocity fields, imposing volume-preserving constraints remains somewhat involved, see e.g. @LucasFidon's paper.

[fvlntn]

Hello @kate-sann5100 and @YipengHu,

Is there any ETA on a tutorial for GlobalNet with unpaired, ddf and label supervision? I'd like to check if my work has no obvious error. I tried by myself and got some results using GlobalNet by using everytime the same atlas for fixed_image and fixed_label. I used as a loss a linear combination of LNCCLoss, DiceLoss and BendingEnergyLoss but don't know which weights are best. Is there any study on that?

Thanks a lot

[YipengHu]

@fvlntn first, not sure we have such a plan as there would be so many permutations with the options of labels, losses and transformation etc ;) but do keep us updated with your experiments with any interesting findings or bugs! As for the weight between these losses, it is a good question and i'd say very much application-dependent (also implementation-dependent as well). For example, we use a very high (>10) weight on BEL on some of our multimosal registration where image similarity (e.g. LNCC) does not always work robustly. But for those with many good segmentation available, label-driven applications, it might need to weight less to allow more local deformation to be learned. Best way is perhaps experiment with your own data (i'm afraid).

[fvlntn]

@YipengHu Thanks for the answer, I meant a tutorial for unpaired registration and a different one for GlobalNet, not necessarily the one I said in my first comment. (As you mentioned in https://github.com/Project-MONAI/tutorials/issues/134#issuecomment-791517929)

Thanks for the information. For the weights, I'm actually experimenting with my data to get the best weights possible for the loss, but it's kinda hard to evaluate registration outputs. Is there any obvious way to evaluate a registration model besides the dice metric? (Loss changes since the weights change everytime).

Last question: Using GlobalNet (Affine), is it normal for BEL to be around 5e-12 ? For both training and validation, BEL (Weight 1) is between 4.5e-12 and 5e-12 which is really low compared to LNCC (Weight 1) between -0.92 and -0.93 and Dice Loss (Weight 1) between 0.4 and 0.3 for instance on my data.

Thanks,

[YipengHu]

@fvlntn :) good catch. i'm afraid we (sort of) decided not go ahead with those suggestions. correct me if i'm wrong here @kate-sann5100 one of the limitations we had was the limited publicly available data which do not allow us to prove/disapprove large useful registration networks. If getting things "just working" for those data, it could even be misleading, for example in the pre-configured hyperparamters that would almost definitely be overfitting the 10-20 training images.

re: registration evaluation - not sure i can help here, it is very much an open research question. Happy to discuss further with more details from your application/data etc.

re: GlobalNet with BEL - GlobalNet is using affine as the transformation model at the moment, BEL will not be that useful, e.g. BE is invariant to translation and rotation. Although numerically you might get some positive values to penalise but i would just set BE weight to 0. We could disable the BEL entirely when GlobalNet is used, but it may be too restrictive for those wanted to try something experimental, e.g. predicting other low-dimensional transformation parameters.

[fvlntn]

Alright, thanks alot for your time and your explanations!