search

dome272 / Diffusion-Models-pytorch

Pytorch implementation of Diffusion Models (https://arxiv.org/pdf/2006.11239.pdf)
Apache License 2.0
1.11k stars 256 forks source link

The model learns very badly #17

Open nikuson opened 1 year ago

nikuson commented 1 year ago

Good model results are very often mixed with random noise and getting good results is not constant even after many hours of training. Maybe someone has found a solution to what needs to be changed in the model?

PlantUser commented 1 year ago

The random initalization of the timesteps in sample_timesteps(self, n) is not ideal.We do not have control over the randomness of the timestep,hence some timesteps might not be sampled ,thus resulting to the model not being trained to predict noise on this time excluded steps. Solution: Try look for a way to control the randomness of the sample times steps