train loss on cifar10 dataset is nan

VinAIResearch / WaveDiff

Official Pytorch Implementation of the paper: Wavelet Diffusion Models are fast and scalable Image Generators (CVPR'23)

GNU Affero General Public License v3.0

366 stars 28 forks source link

train loss on cifar10 dataset is nan #7

Closed diadestiny closed 1 year ago

diadestiny commented 1 year ago

Thanks for your work! Why is the loss training on cifar10 dataset nan? I just download the code and run the script (bash run.sh cifar10 train 1)

hao-pt commented 1 year ago

hi there, after cloning this repo for quick check, I experience no nan loss as in your case. However, this issue might be due to the environment setting. I strongly recommend you use the same version of PyTorch 1.10.0 as ours to avoid any unexpected cases.

Amarkr1 commented 1 year ago

Indeed, I reverified this. A different PyTorch version other than 1.10.0 results in nan. Not sure what leads to this.