Proposals: Registration and Segmentation with Elastix & MONAI Label

ntatsisk commented 1 year ago

Hi project-week community,

We have discussed a bit with Andres Diaz-Pinto (@diazandr3s) on possible projects for the upcoming Project Week that fall under the general title of 3D Medical Image Registration and Segmentation using Elastix and MONAI Label.

Proposals

Here are our tentative proposals for the moment:

Proposal 1

Train a single modality MONAI Label models on Elastix-aligned brain images (T1, T2, FLAIR, etc) using SynthSeg (https://github.com/BBillot/SynthSeg) as the source of annotated datasets - For Nomal brains

SynthSeg is a tensorflow-based deep learning segmentation tool for brain MRIs. It consists of a generative network that produces the synthetic images and a 3D U-Net trained to do the segmentation. The only input (training data) is the training labels so no real images are used.

We will use SynthSeg to produce annotations as “ground truth” on a publicly available dataset like BRATS (multimodal + non-healthy brains) or OASIS (temporal/monomodal + healthy brains). Elastix will be used for the co-registration of the different modalities or temporal images and achieve segmentation via registration.

Proposal 2

Train a MONAI Label model using the raw BRATS dataset (https://www.kaggle.com/competitions/rsna-miccai-brain-tumor-radiogenomic-classification/data). Elastix will be used to co-register the 4 modalities.

This is a classification task. Hypothesis: Registration with elastix might help the classification accuracy. So, we can compare the classification result with and without pre-alignment.

Proposal 3

Extend the whole brain segmentation model available in the Model Zoo (https://github.com/Project-MONAI/model-zoo/tree/dev/models/wholeBrainSeg_Large_UNEST_segmentation)

The data used for the training were registered affinely in the MNI305 space. Hence, elastix can be used to also register any data used for inference in the same space. We could also store all the result transform parameters so that the users could just do the resampling directly without registering again (this holds true for the traning data - unseen data used for inference should still need to be registered).

Proposal 4

Compare registration performance between cross-modal registration (CT-MRI) versus intra-modal registration via synthesised MRI (MRI_syn - MRI). MONAI for the synthesis and elastix for the registration. What would a suitable dataset be?

Proposal 5

Train MONAI Label model for automatic landmark identification in e.g. lung images (dataset: https://med.emory.edu/departments/radiation-oncology/research-laboratories/deformable-image-registration/index.html) . Landmarks can be used either to assist registration with elastix OR elastix can be used to validate the landmark accuracy. 3D Slicer can be used to visualize the landmarks and ease the qualitative evaluation.

Relevant resources

Elastix repo: https://github.com/SuperElastix/elastix
itk-elastix (python wrapping): https://github.com/InsightSoftwareConsortium/ITKElastix
MONAI Label: https://monai.io/label.html

Looking forward to your feedback, declaration of interest, ideas or proposals. Projects can be adjusted, added or removed. Thanks! 🙏

pieper commented 1 year ago

This sounds great, and I'd be interested in being involved.

ntatsisk commented 1 year ago

Closing this proposal in favour of the project issue #713

NA-MIC / ProjectWeek