search

vt-vl-lab / Guided-pix2pix

[ICCV 2019] Guided Image-to-Image Translation with Bi-Directional Feature Transformation
https://filebox.ece.vt.edu/~Badour/guided_pix2pix.html
Other
194 stars 30 forks source link

Evaluate about texture transfer #6

Closed mxkai closed 4 years ago

mxkai commented 4 years ago

Hi, thanks for your great work. Could you please provide the evaluated code for texture transfer？

BadourAlBahar commented 4 years ago

You can follow the explained evaluation of texture transfer. For every sketch in the test set, we generate 10 random sized ground truth texture patches using the texture patch generation algorithm from TextureGAN (https://github.com/janesjanes/Pytorch-TextureGAN). When the evaluation is done, the results of bFT will be saved in ./texture_results and the results of textureGAN in ./texture_results/textureGAN. You can then use the saved images to compute the LPIPS and FID metrics.

mxkai commented 4 years ago

Thanks for your advices, and I will try it.

mxkai commented 4 years ago

I run the evaluate.py with the following problems: anaconda36/lib/python3.6/site-packages/skimage/color/colorconv.py:985: UserWarning: Color data out of range: Z < 0 in 1 pixels warn('Color data out of range: Z < 0 in %s pixels' % invalid[0].size). I calculated the FID score of 10 randomly generated images on the shoes dataset, and the mean FID score is 125.0890. The score is higher than the value in the paper,is this a reasonable result?

BadourAlBahar commented 4 years ago

The reported FID result for the shoes dataset is 121.241 for 10 random sized ground truth texture patches per test image. You can see the texture transfer results in the project page.

mxkai commented 4 years ago

Thanks for your experimental data, I get a reasonable FID score that is 121.1806. I calculated the FID score of per test image separately and then averaged them, which leads to the wrong results. It is correct to calculate the FID score of all test images directly, and I will regenerate the random sized ground truth texture patches to ensure consistent input and calculate the FID score in this way.