search

hutaiHang / Faster-Diffusion

[NeurIPS 2024] Official implementation of "Faster Diffusion: Rethinking the Role of UNet Encoder in Diffusion Models"
https://arxiv.org/abs/2312.09608
Apache License 2.0
302 stars 19 forks source link

The insights about adding x_T #9

Open zen-d opened 10 months ago

zen-d commented 10 months ago

@hutaiHang Thanks for your great work. In the paper, it is said that "prior noise injection" is to enrich the texture information, but not much explanation about it. Could you please provide more motivation or insights about this technique, specifically this line of code https://github.com/hutaiHang/Faster-Diffusion/blob/main/utils_sd.py#L323, why it could boost the texture detail representation? Thank you.

Zhuoyao1012 commented 8 months ago

Actually I think "prior noise injection" can not always work. I have tried Faster Diffusion for SDXL, the prior noise injection would hurt the quality a lot.

zen-d commented 8 months ago

@Zhuoyao1012 Thanks for sharing your experience empirically. Theoretically, the motivation and solution seem also confusing to me.