CompVis / latent-diffusion

High-Resolution Image Synthesis with Latent Diffusion Models
MIT License
11.7k stars 1.52k forks source link

Confusion about get_learned_conditioning() #351

Open SoHotLB opened 7 months ago

SoHotLB commented 7 months ago

Confusion about get_learned_conditioning() function https://github.com/CompVis/latent-diffusion/blob/main/ldm/models/diffusion/ddpm.py#L551

def get_learned_conditioning(self, c):
    if self.cond_stage_forward is None:
        if hasattr(self.cond_stage_model, 'encode') and callable(self.cond_stage_model.encode):
            c = self.cond_stage_model.encode(c)
            if isinstance(c, DiagonalGaussianDistribution):
                c = c.mode()
        else:
            c = self.cond_stage_model(c)
    else:
        assert hasattr(self.cond_stage_model, self.cond_stage_forward)
        c = getattr(self.cond_stage_model, self.cond_stage_forward)(c)
    return c

If AutoencoderKL was used in the first stage, then the get_learned_conditioning() function should have been c = c.sample() Why is it c = c.mode() in the code?

yanjin340121 commented 5 months ago

hi I feel like the problems we encountered are somewhat similar https://github.com/CompVis/latent-diffusion/issues/364