search

zju-pi / diff-sampler

An open-source toolbox for fast sampling of diffusion models. Official implementations for our [CVPR-2024, ICML-2024] papers
Apache License 2.0
165 stars 8 forks source link

SD, ADM, LDM也都单卡跑吗？batchsize好小啊，lr， king也用一样的吗？ #8

Closed YuChen-Liang closed 4 months ago

YuChen-Liang commented 4 months ago

我未能复现LSUNbedroom LDM的结果。您能提供一下DPM++求解器的sample参数吗

YuChen-Liang commented 4 months ago

设置predict_x0=False反而work了，这很奇怪。看起来实现有一些问题

zhyzhouu commented 4 months ago

1.我们最多使用了四张A100 2.Batch size: 主要依照显存来设置。四卡下，SD/ImageNet256/Bedroom取batch_size=32，Bedroom_LDM/ImageNet64/FFHQ64取batch_size=64，CIFAR10取128。 3.Leaning rate: 统一取5e-3 4.kimg: 出于时长考虑，SD取5；其他统一取10。 5.对于SD/LDM/ImageNet256，我们默认取predict_x0=False，lower_order_final=True。对NFE=4下的Bedroom_LDM，lower_order_final=False效果更好 6.等目前的项目处理完后，我会在文档中更新主要实验设置

1.We use up to 4 A100 GPUs. 2.We set batch_size arroding to the graphics memory. Using 4 GPUs, we set 32 for SD/ImageNet256/Bedroom, 64 for Bedroom_LDM/ImageNet64/FFHQ64 and 128 for CIFAR10. 3.The learning rate is 5e-3 across all experiments. 4.kimg: mostly 10. Only for SD we use 5 considering the training time. 5.For SD/LDM/ImageNet256, we use predict_x0=False and lower_order_final=True by default. We find lower_order_final=False better for 4-NFE Bedroom_LDM. 6.Currenly working on other projects. Afterwards, I will update the main experimental settings in the document.

YuChen-Liang commented 4 months ago

通过与DPMv3的代码对照，LDM是不使用thresholding的，predict_x0=True效果会好一些。