Does this work with CLIP guidance for stable diffusion?

[eware-godaddy]

Wondering if this code base means we can use CLIP guidance for generation instead of the classifier free guidance in the regular model?

[Jack000]

kind of, you can backprop the gradients through the vae but it uses a lot of vram and doesn't work that well in my experience. Ideally there should be a latent CLIP trained on the LDM embeddings instead of pixels.