search

eclipse-t2i / lambda-eclipse-inference

Official PyTorch implementation of "λ-ECLIPSE: Multi-Concept Personalized Text-to-Image Diffusion Models by Leveraging CLIP Latent Space"
https://eclipse-t2i.github.io/Lambda-ECLIPSE/
MIT License
43 stars 5 forks source link

about the decoder #4

Open ysm2000 opened 3 months ago

ysm2000 commented 3 months ago

Thanks for your nice work! I have a question, If I replace the decoder with the SD series, will it affect the final generation performance?

Maitreyapatel commented 3 months ago

Hi @ysm2000, Yes, it may affect it. We observed that the following parts play crucial roles:

  1. Another decoder should be conditioned on the given CLIP embeddings used by our model.
  2. Models like stable unclip do this, but due to insufficient training of these models, they do not perform well.

Therefore, Kandinsky v2.2 is the ideal candidate.