search

hutaiHang / Faster-Diffusion

[NeurIPS 2024] Official implementation of "Faster Diffusion: Rethinking the Role of UNet Encoder in Diffusion Models"
https://arxiv.org/abs/2312.09608
Apache License 2.0
302 stars 19 forks source link

This algorithm seems to only work with standard SD？ #7

Open jianyuheng opened 10 months ago

jianyuheng commented 10 months ago

I tried it on the community model and got poor generation results.

hutaiHang commented 10 months ago

I tried it on the community model and got poor generation results.

Can you provide more information? What kind of model? We have tested our method on SDXL, SD 2.0, and others, and it works well. We only need to adjust the hyperparameter `key timestep｀，and we will release all the code later.

hutaiHang commented 10 months ago

I tried it on the community model and got poor generation results.

We tested our approach in the community model Realistic Vision V6.0 B1, and achieved good results when uniformly setting the keytime at a 1:4 ratio, as shown in the figure below:

image

The prompt used was:"adorable Corgi wearing a hat is sitting next to a Christmas tree."

jianyuheng commented 10 months ago

Sorry for late reply. I tried rev-v12 inference with Faster Diffusion and got bad results.

image
hutaiHang commented 10 months ago

Sorry for late reply. I tried rev-v12 inference with Faster Diffusion and got bad results. image

Thanks for your feedback. We just tested the rev-v12 model. When uniformly setting the keytime at a 1:4 ratio , it stil works well, as shown in the following figure. So there may be some hyperparameter configuration errors for you. Please refer to our latest updated code image

jianyuheng commented 10 months ago

@hutaiHang Sorry I left out some details. My scheduler is EulerAncestralDiscreteScheduler and I guess this is the key to not working.