This draft PR adds support for noise perturbation as described in the paper Input Perturbation Reduces Exposure Bias in Diffusion Models.
The authors found "without affecting the recall and precision, the proposed input perturbation leads to a significant improvement in the sample quality while reducing both the training and the inference times."
They adjust the noising process as follows:
I'm running this with karras/soft-min-snr right now but haven't implemented/tested the other Denoiser classes. Is this something you'd be interested in upstreaming if I finished and tested it?
https://arxiv.org/pdf/2301.11706.pdf
This draft PR adds support for noise perturbation as described in the paper Input Perturbation Reduces Exposure Bias in Diffusion Models.
The authors found "without affecting the recall and precision, the proposed input perturbation leads to a significant improvement in the sample quality while reducing both the training and the inference times."
They adjust the noising process as follows:
I'm running this with karras/soft-min-snr right now but haven't implemented/tested the other Denoiser classes. Is this something you'd be interested in upstreaming if I finished and tested it?