search

openai / improved-diffusion

Release for Improved Denoising Diffusion Probabilistic Models
MIT License
3.27k stars 488 forks source link

FID not matching on CIFAR10 #106

Open akrlowicz opened 1 year ago

akrlowicz commented 1 year ago

I downloaded CIFAR10 500K checkpoint from readme, sampled it for 10k images and my FID points to > 200. I tested few different implementations of FID calculations and am comparing it to CIFAR10 50k training data. It's ridiculously high, nowhere near 3 and my models trained on CIFAR10 also show high FID after many iterations. What could be wrong in this case?

loki-veera commented 1 year ago

Even I am facing the same issue, computing FID with dims 64 for InceptionNet gives me FID of 1.52

bigwahaha commented 1 year ago

I downloaded CIFAR10 500K checkpoint from readme, sampled it for 10k images and my FID points to > 200. I tested few different implementations of FID calculations and am comparing it to CIFAR10 50k training data. It's ridiculously high, nowhere near 3 and my models trained on CIFAR10 also show high FID after many iterations. What could be wrong in this case?

The FID I got was ridiculously high too.Have you figured out what the problem is?

loki-veera commented 1 year ago

FYI: I was able to get the FID of 4.41 for hybrid loss and 7.88 for VLB loss (3.19 and 11.47 as stated in paper) with dividing the samples with 255. so they are float type instead of uint8 and also the original dataset is saved in JPEG format.