search

lucidrains / DALLE2-pytorch

Implementation of DALL-E 2, OpenAI's updated text-to-image synthesis neural network, in Pytorch
MIT License
11.17k stars 1.09k forks source link

How should i understand useing self conditioning in DiffusionPriorNetwork which is Hinton's group's new ddpm technique. #273

Open SELECT-FROM opened 1 year ago

SELECT-FROM commented 1 year ago

If self.self_cond is true , how should I understand learned_queries .

    if self.self_cond:
        learned_queries = torch.cat((image_embed, self_cond), dim=-2)
lucidrains commented 1 year ago

@SELECT-FROM that doesn't look right :disappointed: i believe it should be correct now

the logic should have been to include the self conditioning token just before the learned queries

SELECT-FROM commented 1 year ago

@lucidrains Thank you for your reply. I have another question is in what scenario or requirement should self conditioning be set to true? The default value false is always used in the init method of DiffusionPriorNetwork

lucidrains commented 1 year ago

@SELECT-FROM the whole field is very empirical

i would recommend just running a comparison test with that one flag self_cond toggled and go by the results you see