Color convergence problem on custom data

[wileewang]

Hi! Thanks for your great work. I want to ask why it is difficult to converge on some custom datasets. These dataset have a very uniform hue and lighting (e.g., Cityscape, GTA5) , but color information is learned almost at the end of the learning process. In other words, I have been training for a long time but the generated images still have a relatively large difference in color. This seems to be very easy to learn in the beginning for other generative models. Could you give some hints about this probelm ? Or any suggestions about hyper-parameter tuning for such a dataset (Cityscape , GTA5)?

[lulubbb]

Hi bro, I meet the same problem, I want to traning improve diffusion model on my custom pedestrain dataset, but the sample pedestrain result have not clear surface and wrong pose. Could you give some tips abount this problem? And I also want traning diffusion model on CityScape dataset.

[wileewang]

For my experience of training Cityscape, you can select two modes of U-net, i.e., predicts noise or predicts the clean image, and the one predicts noise usually requires more model capacity to learn this dataset.

[miaoYuanyuan]

@wileewang I meet the same problem . I can't understand your means "select two models of U-net", could you give more details ?

[wileewang]

Hi, the u-net of diffusion model can be parameterized to predict the noise or predict the clean image.

[smy1999]

Have you solved the problem? I've got a same one.

[zy-charon]

@wileewang May I ask how to use unet to directly predict the image?

[ONobody]

@wileewang Excuse me, when training your own data set, when the number of categories is different, where can the code be adjusted accordingly?