SHI-Labs / Prompt-Free-Diffusion

Prompt-Free Diffusion: Taking "Text" out of Text-to-Image Diffusion Models, arxiv 2023 / CVPR 2024
https://arxiv.org/abs/2305.16223
MIT License
728 stars 36 forks source link

[Question] Pre-computed unconditional embeddings #4

Closed BlenderNeko closed 1 year ago

BlenderNeko commented 1 year ago

I noticed that the assets folder contains some pre-computed embeddings that function as the unconditional input. Were these computed similarly to how the graido app uses black images to provide the unconditional embedding?

xingqian2018 commented 1 year ago

Although we provide assets/clip_ug.pth, we didn't actually use that. The UG for prompt-free diffusion is all zeros vectors. But prompt-free is totally fine with traditional UG (i.e. negative prompts), you may try use it. Our experience shows little improvement.

The anime app, however, assets/anime_ug.pth provided improvement in results, so we apply that in the app.