search

cvlab-stonybrook / PathLDM

Official Code for PathLDM: Text conditioned Latent Diffusion Model for Histopathology (WACV 2024)
33 stars 3 forks source link

Training unconditional LDM gives RuntimeError #14

Closed MeesMeuwissen closed 7 months ago

MeesMeuwissen commented 7 months ago

Hi, I am trying to train an unconditional LDM by setting the cond_stage_config to __is_unconditional__. This ensures that context is None when running through the Unet. My issue is that this gives rise to a RuntimeError, namely "linear(): input and weight.T shapes cannot be multiplied (1024x384 and 512x384)" which happens in the fifth module in input_blocks. Adding in dummy context solves this error. Is this the preferred way of training an unconditional model? Or is there any other setting I should change to support unconditional models? Thanks

srikarym commented 7 months ago

Using a dummy context should be fine. Even in the current setting, since we use classifier-free guidance, we set the context to a null token 10% of the time.