search

rosinality / stylegan2-pytorch

Implementation of Analyzing and Improving the Image Quality of StyleGAN (StyleGAN 2) in PyTorch
MIT License
2.75k stars 624 forks source link

Performance gap compared with the official TensorFlow implementation #91

Open laoreja opened 4 years ago

laoreja commented 4 years ago

Hi,

Thanks for the great repo!

I saw that you mentioned "I have trained the 256px model on FFHQ 550k iterations. I got FID about 4.5. Maybe data preprocessing, resolution, training loop could made this difference, but currently I don't know the exact reason of FID differences."

May I ask what's the performance of the original TensorFlow repo with the same training setting? From their paper and repo, I only found the FID for 1024x1024 resolution. And I haven't used TensorFlow for many years...

Thank you in advance!

rosinality commented 4 years ago

You can get 3.81 FID for ffhq at 256px. (from https://arxiv.org/abs/2006.06676)

cientgu commented 4 years ago

You can get 3.81 FID for ffhq at 256px. (from https://arxiv.org/abs/2006.06676)

Yes, but I found in the tensorflow implementation, there are implementation bugs on compute the FID. They compute FID between 50k generated images at 256x256 and 50k ground truth images at 1024x1024, which is significant lower than compute them all on 256x256. I suggest to use the pytorch-fid(https://github.com/mseitzer/pytorch-fid) on both 50k 256x256 images to get the correct FID score.

shoutOutYangJie commented 3 years ago

You can get 3.81 FID for ffhq at 256px. (from https://arxiv.org/abs/2006.06676)

Yes, but I found in the tensorflow implementation, there are implementation bugs on compute the FID. They compute FID between 50k generated images at 256x256 and 50k ground truth images at 1024x1024, which is significant lower than compute them all on 256x256. I suggest to use the pytorch-fid(https://github.com/mseitzer/pytorch-fid) on both 50k 256x256 images to get the correct FID score.

Do you know how to evaluate this repo? I can't find the test code.

cientgu commented 3 years ago

You can get 3.81 FID for ffhq at 256px. (from https://arxiv.org/abs/2006.06676)

Yes, but I found in the tensorflow implementation, there are implementation bugs on compute the FID. They compute FID between 50k generated images at 256x256 and 50k ground truth images at 1024x1024, which is significant lower than compute them all on 256x256. I suggest to use the pytorch-fid(https://github.com/mseitzer/pytorch-fid) on both 50k 256x256 images to get the correct FID score.

Do you know how to evaluate this repo? I can't find the test code.

just use pytorch-fid(https://github.com/mseitzer/pytorch-fid)