Potential feature: One-shot generative models based on single training image

[RichardObi]

For use-cases where there are very few training images (e.g. perhaps rare diseases), it could be nice to augment the data using one-shot generative models. Could this be something of interest to have in MONAI GenerativeModels? Example models include sinfusion, SinGAN, OneShotGAN, ConSinGAN

[Warvito]

That sounds like a good idea, @RichardObi ^^ I need to get more familiar with these methods yet. For our package, which one of them do you think we start with? Which one looks more promising for medical data (2D and 3D)?

[RichardObi]

Cool :) Often these methods learn from patches of the training image or train multiple generators that iteratively scale the training image up with some variation in each step.

• SinGAN is the most widely used and cited. So far, it has been used to generate 2D medical imaging data (ref1, ref2, ref3, ref4).
• The other one-shot generative models such as OneShotGAN and ConSinGAN extend upon SinGAN. I could not find an application of them to medical images yet.
• sinfusion is based on a diffusion model rather than a GAN. It is still quite recent (Nov 2022) and (therefore) has not yet been tested widely with medical images. In the paper, it is also applied to video data and perhaps the video methodology could guide its application to 3D medical data. Therefore, sinfusion could be a great addition to GenerativeModels and perhaps some of the diffusion model training and inference code that is already in GenerativeModels could be reused to integrate sinfusion.