search

dome272 / Diffusion-Models-pytorch

Pytorch implementation of Diffusion Models (https://arxiv.org/pdf/2006.11239.pdf)
Apache License 2.0
1.11k stars 256 forks source link

img2img #19

Closed liuzysy closed 1 year ago

liuzysy commented 1 year ago

Does anyone have some experience on img2img by using diffusion model?

dome272 commented 1 year ago

If you want to use CLIP. You probably don't need to change much. Just making the model a bit bigger and then instead of conditioning on learned class embeddings as I do in the code, you just provide in the CLIP embedding. CLIP embeddings are also just a vector, so same as class embeddings. Does that help?