search

phizaz / diffae

Official implementation of Diffusion Autoencoders
https://diff-ae.github.io/
MIT License
863 stars 130 forks source link

Sampling without noise during training #46

Open anthony-mendil opened 1 year ago

anthony-mendil commented 1 year ago

First of all, thanks for your great work! I am trying to understand how to properly use deterministic noising and denoising. From your code (in particular the file diffae/diffusion/base.py) I get that there are function to do this: ddim_reverse_sample_loop for the deterministic noising and ddim_sample_loop for the deterministic denoising. However, it seems like these are never used during training but only after training.

For the denoising during training, only the function q_sample is used and the noise parameter is always set to none so that gaussian noise is added.

So it seems like during training the denoising process is not deterministic but then after training you use the deterministic functions mentioned before.

Is this observation correct? I find it hard to understand the reason for this difference. Is it desired that the generation is non-deterministic during training and only deterministic after training?

More precisely, I was expecting that the ddim_reverse_sample_loop would also be used during training so that the latent variables of the same images are also equal during training (due to the deterministic property of ddim).

I would very much appreciate it you could clear up my confusion. Thanks in advance, Anthony Mendil.

anthony-mendil commented 1 year ago

If I understand whats written in your paper correctly, this is intentional. However, I am still unsure about the reason and it is not explained in the paper. I would appreciate if you could explain the thoughts behind this.

phizaz commented 1 year ago

So it seems like during training the denoising process is not deterministic but then after training you use the deterministic functions mentioned before.

Your observation is correct. This is how you train a DPM and it follows theoretically from the DPM model settings. I suggest you take a look at DPPM's paper (https://arxiv.org/abs/2006.11239) and DDIM's paper (https://arxiv.org/abs/2010.02502).

anthony-mendil commented 1 year ago

Thanks for the response! I will take a look at the papers.