search

LTH14 / rcg

PyTorch implementation of RCG https://arxiv.org/abs/2312.03701
MIT License
785 stars 36 forks source link

Some question about FID evaluation #35

Closed mapengsen closed 3 months ago

mapengsen commented 3 months ago

I see that you use the ADM assessment approach:

In ADM (https://github.com/openai/guided-diffusion/blob/main/evaluations/README.md) image

They say pre-computed statistics over the whole dataset, but when i load ImageNet 256x256: [reference batch](https://openaipublic.blob.core.windows.net/diffusion/jul-2021/ref_batches/imagenet/256/VIRTUAL_imagenet256_labeled.npz)

image

but get 10000 image, not whole dataset, Is rcg also calculated in this way? I don't understand the 10000 here, could you please help me? Thank you very much!

LTH14 commented 3 months ago

This also confused me for a while. In the ADM-precomputed reference batch, the pre-computed statistics for FID evaluation are just 'mu' and 'sigma'. The 10000 images in the batch are only for precision and recall computation, and not used for FID.

mapengsen commented 3 months ago

yes, 'mu', 'sigma', is easy to understand, but 'mu_s', 'sigma_s', don't know what it is? And how to calculate, they do not provide the relevant code.

LTH14 commented 3 months ago

I remember it is for sFID.

mapengsen commented 3 months ago

I see. I am also confused here for a long time, really thank you very much :pray: :pray: :pray: