search

w86763777 / pytorch-ddpm

Unofficial PyTorch implementation of Denoising Diffusion Probabilistic Models
Do What The F*ck You Want To Public License
506 stars 62 forks source link

Pretrained example: evaluate model and calculate FID score #15

Closed jS5t3r closed 1 year ago

jS5t3r commented 1 year ago

How many samples for the FID is good? As standard is the whole cifar is used.

The computation of FID is slowly because running on CPU(). I don't get any FID because the FID calculation takes hours... I am using an HPC and I should have a lot of CPU is there a faster way to calculate the FID?

w86763777 commented 1 year ago

To the best of my knowledge, both the number of generated images and the reference statistics of the dataset can significantly influence the FID results. It is recommended to generate 50,000 images to obtain a reasonable result that is consistent with the DDPM paper.

However, generating 50,000 images can take a considerable amount of time due to the large number of denoising steps involved. In my experience, running a single evaluation on a single RTX2080Ti GPU takes several hours (I do not recall the exact time). Therefore, I recommend that you perform the evaluation on a machine with at least one GPU.

jS5t3r commented 1 year ago

How long does take the FID calculation after the samples are generated?

w86763777 commented 1 year ago

Just CPU computation is needed, and the required time usually takes less than one minute.

jS5t3r commented 1 year ago

OK. It gets stuck in computation in my case.

jS5t3r commented 1 year ago

OK. I can reproduce now your output. I set up the exact same environment. Unfortunately, I am not able to use this setup an Nvidia A100 and need to use a Titan V (12GB).