search

modelscope / scepter

SCEPTER is an open-source framework used for training, fine-tuning, and inference with generative models.
https://github.com/modelscope/scepter
Apache License 2.0
405 stars 22 forks source link

First stage training config (Autoencoder) #30

Closed sahilqure closed 4 months ago

sahilqure commented 5 months ago

I understand that full finetuning sdxl without the first stage on natural images works precisely but for images of other domains like (CT, MRI, and stuff), first-stage training of autoencoder is required.

jiangzeyinzi commented 5 months ago

In the generic diffusion-based text-to-image generation model, the VAE is typically trained independently with the addition of reconstruction loss and GAN loss. Currently, we do not have the corresponding implementation or configuration files, and contributions are welcome.