search

lucidrains / DALLE2-pytorch

Implementation of DALL-E 2, OpenAI's updated text-to-image synthesis neural network, in Pytorch
MIT License
11.15k stars 1.09k forks source link

Questions about inpaint training code #249

Open universewill opened 2 years ago

universewill commented 2 years ago

Hi, i don't quite understand the inpainting training framework.

  1. Is only the decoder need to be trained?
  2. What does the "mock_image_embed" mean below? Is this embed need to be generated from clip and diffusion prior model?
inpainted_images = decoder.sample(
    image_embed = mock_image_embed,
    inpaint_image = inpaint_image,    # just pass in the inpaint image
    inpaint_mask = inpaint_mask       # and the mask
)
zhangchen234 commented 2 years ago

I have the same question.