Allow LDM model to train on non-quantized encoded outputs of the VQ-VAE

Project-MONAI / GenerativeModels

MONAI Generative Models makes it easy to train, evaluate, and deploy generative models and related applications

Apache License 2.0

598 stars 86 forks source link

Allow LDM model to train on non-quantized encoded outputs of the VQ-VAE #473

Closed virginiafdez closed 6 months ago

virginiafdez commented 6 months ago

Function _encode_stage_2inputs of the VQ-VAE outputs the quantised latent representations. Since _decode_stage_2outputs quantises the input anyway, we could have a flag in _encode_stage_2inputs to allow for the user to train an LDM on non-quantised representations.