Add a CycleGAN with mutual information as a consistency loss

[dyollb]

Is your feature request related to a problem? Please describe. Provide an style transfer model to learn to generate synthetic CT from MRI, or T2 from T1, or e.g. T1 from on scanner to T1 from a different scanner/site without loosing anatomical/geometric information.

Describe the solution you'd like Add a CycleGAN with mutual information as a consistency loss, e.g. implementing https://arxiv.org/abs/1912.08061

Modanwal, Gourav, Adithya Vellal, and Maciej A. Mazurowski. "Normalization of breast MRIs using Cycle-Consistent Generative Adversarial Networks." Computer Methods and Programs in Biomedicine (2021): 106225.

Describe alternatives you've considered It would be nice to have a tutorial illustrating different losses.

Additional context

• predict CT without inducing radiation in patients
• segment bones from MRI as if you had segmented used CT
• adapt data from different site to look similar to data used to train a unet

[wyli]

thanks for the request, as a reference, are you aware of any open-source implementation in any programming language?

[dyollb]

For the original un-paired cyclegan (and paired pix2pix): https://github.com/junyanz/pytorch-CycleGAN-and-pix2pix

pix2pix works nicely if the images are provided as registered pairs. But quality deteriorates if registration is not good.

I did not find any implementation of a cyclegan using the mutual information loss, or other losses (e.g. gradient consistency loss) developed to preserve the geometry/boundaries.

The paper by Modanwal et al uses Mutual-Information-Neural-Estimation (MINE) to compute mutual information (paper, code)