How to calculate FID? Any bash scripts?

[sumorday]

The celebahq_stat.npy file for FID testing is already in the pytorch_fid folder, and the generated images are in ../saved_info/latent_flow/celeba_256/celeb256_f8_adm/image_epoch_20.png.

Should I run !bash ../bash_scripts/run_test_ddp.sh ../test_args/celeb256_adm.txt

or just !bash ../bash_scripts/run_test_ddp.sh?

any bash scripts examples for running FID calculation? Thank you in advance.

[sumorday]

!bash ./bash_scripts/run_test_ddp.sh test_args/celeb256_adm.txt

[sumorday]

Traceback (most recent call last): File "/content/drive/MyDrive/LFM/test_flow_latent_ddp.py", line 263, in main(args) File "/content/drive/MyDrive/LFM/test_flow_latent_ddp.py", line 59, in main ckpt = torch.load("./saved_info/latentflow/{}/{}/model{}.pth".format(args.dataset, args.exp, args.epoch_id)) File "/usr/local/lib/python3.10/dist-packages/torch/serialization.py", line 986, in load with _open_file_like(f, 'rb') as opened_file: File "/usr/local/lib/python3.10/dist-packages/torch/serialization.py", line 435, in _open_file_like return _open_file(name_or_buffer, mode) File "/usr/local/lib/python3.10/dist-packages/torch/serialization.py", line 416, in init super().init(open(name, mode)) FileNotFoundError: [Errno 2] No such file or directory: './saved_info/latent_flow/celeba_256/celeb256_f8_adm/model_450.pth'

[sumorday]

CelebA 256x256 maybe run_test.sh ,this file? not run_test_ddp.sh ?

!bash ./bash_scripts/run_test.sh test_args/celeb256_adm.txt

FileNotFoundError: [Errno 2] No such file or directory: './saved_info/latent_flow/celeba_256/celeb256_f8_adm/model_450.pth'

[sumorday]

should I change the EPOCH_ID=450 to EPOCH_ID=30 if i only running it 30 times? otherwise the test.sh can not find the 450.pth file?

[hao-pt]

Hi,

Basically, run_test.sh & run_test_ddp.sh are the same where both can be used to generate output samples and calculate FID. The main difference is that run_test_ddp.sh support multi-gpu running for faster image generation.

The error happened due to the mismatch of arg EPOCH_ID. You should adjust it to corresponding epoch id of your trained model such as EPOCH_ID=30 in your case.

If you already generated images in a particular folder, you could use this script for direct FID calculation: python pytorch_fid/fid_score.py ./pytorch_fid/celebahq_stat.npy path_to_output_dir

[sumorday]

Thank you so much!

!python pytorch_fid/fid_score.py ./pytorch_fid/celebahq_stat.npy ./saved_info/latent_flow/celeba_256/celeb256_f8_adm_saved

_pickle.UnpicklingError: pickle data was truncated

Do I have to finish running, for example, 600 or 450 times, before I can test FID? My folder is as shown in the picture. For the CelebA 256x256 dataset, it seems that a new generated image is recorded every 5 steps. I can't seem to test FID.

[sumorday]

Downgrade scipy to 1.11.1 solved the problem for me. pip install scipy==1.7.3

Not so sure if the FID value is correctly or not.

!python pytorch_fid/fid_score.py ./pytorch_fid/celebahq_stat.npy ./saved_info/latent_flow/celeba_256/celeb256_f8_adm

/usr/local/lib/python3.10/dist-packages/torchvision/models/_utils.py:208: UserWarning: The parameter 'pretrained' is deprecated since 0.13 and may be removed in the future, please use 'weights' instead. warnings.warn( /usr/local/lib/python3.10/dist-packages/torchvision/models/_utils.py:223: UserWarning: Arguments other than a weight enum or None for 'weights' are deprecated since 0.13 and may be removed in the future. The current behavior is equivalent to passing weights=None. warnings.warn(msg) Warning: batch size is bigger than the data size. Setting batch size to data size 100% 1/1 [00:01<00:00, 1.04s/it] FID: 510.65202745674986