search

jychoi118 / ilvr_adm

ILVR: Conditioning Method for Denoising Diffusion Probabilistic Models (ICCV 2021 Oral)
MIT License
417 stars 51 forks source link

Questions regarding evaluation #1

Open Seth-Park opened 3 years ago

Seth-Park commented 3 years ago

Hi,

Thanks for sharing the codebase and the pretrained models! Your work is very impressive :) I just wanted to ask a few questions regarding quantitative evaluation: 1) How many images were used for computing the FID scores in Table 1 of the main paper? 2) How are the reference images that guide the generation process chosen? 3) What codebase did you use to compute the FID scores?

Regards, Seth

jychoi118 commented 3 years ago

Hi, thank you for enjoying our paper.

  1. We used 50k generated samples and 50k random real images from the training set. (Generating 50k images with full 1000 steps takes a long time..)
  2. For FID score, 50k reference images are randomly chosen from the training set.
  3. We followed https://github.com/mseitzer/pytorch-fid for FID scores. I would like to note that our experiments in the paper are conducted with this codebase: https://github.com/rosinality/denoising-diffusion-pytorch . So, we measured FID scores with data pre-processing from this codebase.