Closed AlvL1225 closed 1 year ago
for sampling, 64 and 32 are identical as long as the attention_resolutions are shifted.
32px with attention resolutions 32,16,8 = 64px with attention resolutions 64,32,16
I used 32px mostly because it matches the settings in the Compvis code.
the KL-VAE is (presumably) trained at 256px. It can be used for any resolution >= 256px (smaller images may have issues, based on my testing)
the image size for the training code is set with --actual_image_size - and yes, to match SD it should be set to 512 (all other flags can stay the same). I should probably change this.
Thanks!
I have another question. On what device do you test and how is the speed? (with 256 and 32 settings)
Do you think if I can train on a 24GB device (or multiple 3090s) with half the batch size per gpu?
Sampling and training speed should be identical to the Compvis repo, it's basically the same code.
I don't think it would be possible to train on a 3090 with this code. It just only off by a little bit though, so it might be possible with a bit of work - disable ema and change the training script to pre-compute the LDM embeddings and store on disk.
A better solution might be to use deepspeed cpu offloading similar to this repo: https://github.com/afiaka87/latent-diffusion-deepspeed
Hi Jack, Resolutions for SD official V1.4 (V1.3) model are 512 for kl and 64 for diffusion model but your Readme settings are 256 for kl, and 32 for diffusion model.
Can the two resolution settings match? Thanks!