Task-Induced Pyramid and Attention GAN for Multimodal Brain Image Imputation and Classification in Alzheimer’s Disease

Paper: https://ieeexplore.ieee.org/document/9490307

Summary

In practice, multimodal images may be incomplete since PET is often missing due to high financial costs or availability. Most of the existing methods simply excluded subjects with missing data, which unfortunately reduced the sample size. To address these problems, we propose a task-induced pyramid and attention generative adversarial network (TPA-GAN) for imputation of multimodal brain images. With the complete multimodal images, we build a pathwise transfer dense convolution network (PT-DCN) to gradually learn and combine the multimodal features for final disease classification.

Overview

Installation

This script need no installation, but has the following requirements:

PyTorch 1.11.0 or above
Python 3.5.7 or above

Usage

Import libraries

import os
import cv2
import torch
import numpy as np
from torch import nn
from torch import optim
from torch.nn import functional as F 
from torch.autograd import Variable
from sklearn.metrics import roc_curve, auc
import math
import time

Methods

Stage 1: TPA-GAN for multimodal brain image imputation

Run the following code，we test the models after each epoch of training process. No independent test program is required, we also provide。

train_TPA_GAN.py

Tricks

Pre-train the Task-induced discriminator first. In the training of GAN, the weights of Task-induced discriminator are fixed, while its loss is used to update the parameters of generator, which can help the generator reconstruct the pathological changes.
Adjust the weights of Generator losses adaptively for balance of training.
To start and stop the Standard discriminator and Task-induced discriminator at the right time according to experrience (see source code).
The generalization performance of pre-trained Task-induced discriminatora is vitally important.

Stage 2: PT-DCN for disease classification