Residual Denoising Diffusion Models

paper|arxiv|youtube|blog|中文论文(ao9l)|中文视频|中文博客

This repository is the official implementation of Residual Denoising Diffusion Models.

Note:

1. The current setting is to train two unets (one to estimate the residuals and one to estimate the noise), which can be used to explore partially path-independent generation process.
2. Other tasks need to modify a) [self.alphas_cumsum[t]*self.num_timesteps, self.betas_cumsum[t]*self.num_timesteps]] -> [t,t] (in L852 and L1292). b) For image restoration, generation=False in L120, convert_to_ddim=False in L640 and L726. c) uncomment L726 for simultaneous removal of residuals and noise. d) modify the corresponding experimental settings (see Table 4 in the Appendix).
3. The code is being updated.

Requirements

To install requirements:

conda env create -f install.yaml

Dataset

Raindrop (test-a for test)

GoPro

ISTD

SID-RGB: kexu or download

LOL

CelebA

Training

To train RDDM, run this command:

python train.py

or

accelerate launch train.py

Evaluation

To evaluate image generation, run:

cd eval/image_generation_eval/
python fid_and_inception_score.py path_of_gen_img

For image restoration, MATLAB evaluation codes in ./eval.

Pre-trained Models

Two unets (deresidual+denoising) for partially path-independent generation process

Results

See Table 3 in main paper.

For image restoration:

Raindrop

GoPro

ISTD

LOL

SID-RGB

For image generation (on the CelebA dataset):

We can convert a pre-trained DDIM to RDDM by coefficient transformation (see code).

Citation

If you find our work useful in your research, please consider citing:

@InProceedings{Liu_2024_CVPR,
    author    = {Liu, Jiawei and Wang, Qiang and Fan, Huijie and Wang, Yinong and Tang, Yandong and Qu, Liangqiong},
    title     = {Residual Denoising Diffusion Models},
    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
    month     = {June},
    year      = {2024},
    pages     = {2773-2783}
}

Contact

Please contact Liangqiong Qu (https://liangqiong.github.io/) or Jiawei Liu (liujiawei18@mails.ucas.ac.cn) if there is any question.