search

MIDA-group / CoMIR

Framework for joint representation learning, evaluation through multimodal registration and comparison with image translation based approaches
Other
72 stars 11 forks source link

May I ask if the code provided below can be used for 3D medical image registration, especially for multi-modal registration of lung of CT image and lung of CBCT image #58

Open yuanpengpeng opened 3 months ago

yuanpengpeng commented 3 months ago

May I ask if the code provided below can be used for 3D medical image registration, especially for multi-modal registration of lung of CT image and lung of CBCT image

HBB0517 commented 2 weeks ago

May I ask if the code provided below can be used for 3D medical image registration, especially for multi-modal registration of lung of CT image and lung of CBCT image

请问你用在了三维吗？

joakimlindblad commented 2 weeks ago

Although the provided code is only performing 2D modality translation of multimodal data into CoMIR representations, you may apply the code slice-wise, and then use a 3D image registration on the assembled 3D-CoMIR volumes. We have done so successfully for T1-T2 weighted MRI data in the paper: "Is image-to-image translation the panacea for multimodal image registration? A comparative study" https://doi.org/10.1371/journal.pone.0276196, for which we share a complete and well documented source code repository MultiRegEval, wherein 3D processing is explicitly supported.

I am not aware that we have tried CT-CBCT registration explicitly, but I believe it should not be more difficult than T1-T2. In case you need deformable registration of the computed CoMIRs, you may wish to try INSPIRE, which is a fast deformable extension of the alpha-AMD registration framework.

joakimlindblad commented 2 weeks ago

For more deformable CoMIR (however 2D only), you can also take a look at: L. Nordling, J. Öfverstedt, J. Lindblad, N. Sladoje. Contrastive Learning of Equivariant Image Representations for Multimodal Deformable Registration. In Proceedings of the 20th IEEE International Symposium on Biomedical Imaging (ISBI), IEEE, pp. 1-5, Cartagena, Colombia, April 2023. doi:10.1109/ISBI53787.2023.10230465 Code available at github.com/MIDA-group/CoMIR_INSPIRE