MONAI examples to imitate SynthSR and Synthseg in Freesurfer

[armaneshaghi]

Is your feature request related to a problem? Please describe. I am looking for a tutorial for superresolution models with MONAI. That is models that can learn the mapping between

(1) low-resolution or low-quality (e.g., clinical scans) and (2) semantic segmentation maps (exact opposite of getting semantic segmentation from T1 scans)

with

high-quality scans.

These two requests have different utilities in practice.

There are prior implementations in Freesurfer but I do not know of any similar tutorial using MONAI.

[mingxin-zheng]

Hi @armaneshaghi , this is an interesting topic but I can't estimate how comprehensive it is. Do you want a new model to fulfill the tasks mentioned, or just a tutorial to illustrate if there is an existing model in MONAI to do the tasks?

[armaneshaghi]

@mingxin-zheng I think a model to generate any MRI modality from semantic segmentation maps is a good start. Happy to contribute and dedicate some time to this if we can align strategically, who will do which part: 1) a model from labels to MRI, with randomization for MRI sequence modality and resolution / even randomization for left and right (different modalities in the same MRI to prevent model from learning intensities, instead forcing model to learn the underlying structural anatomy). Freesurfer Segmentation (led by Benjamin Billiot and Eugenio Iglesias) does this using Tensorflow and simple (so powerful) Gaussian Mixture Models. 2) a model from MRI to label, which we can use one of the widely available 3D Unets already in MONAI examples

[wyli]

thanks @armaneshaghi, could you elaborate the first item, a model from labels to MRI, do you mean a network model or sampling from a simple GMM model conditioned on a label map?

[armaneshaghi]

It can be done in both ways, perhaps more relevant to MONAI to have a generative deep neural network model from label maps? Freesurfer uses a simple GMM.

[wyli]

sure, the generative modelling in monai is currently under discussion. At this point if there's a simple GMM like the one in freesurfer might be useful towards 'synthseg workflow in monai', are you interested in implementing the GMM sampling example in MONAI? then we can train seg models on these synthetic data.

[mingxin-zheng]

I also agree with @wyli this can be a new component in MONAI. I am looking forward to the feature you mentioned.