Open windelvin opened 1 year ago
No, the learning rate is fixed as 1e-4 following the setting in DDPM.
Another quick question, Have you tried using 1024*1024 resolution to train the model, Does lr=1e-4 still work? Or the dataset needs to be expanded?
We have not trained the model with 1024 x 1024, which requires huge computation resources.
Have you ever tried to decrease the lr(the default value is 1e-4) for the training procedure of diffusion model? Since the loss converges very quick(less than 1W iters), but the lr keeps constant during the whole training time.