search

Ha0Tang / C2GAN

[ACM MM 2019 Oral] Cycle In Cycle Generative Adversarial Networks for Keypoint-Guided Image Generation
http://disi.unitn.it/~hao.tang/project/C2GAN.html
Other
69 stars 5 forks source link

Get wrong result during test phase #6

Closed sitadivon closed 4 years ago

sitadivon commented 4 years ago

Hi, Hao Tang! I met with difficulties again. Could you help me to figure them out?

I tried to reproduce your experiment but got wrong result. I used the weight you provided to run the network. While the result of SSIM was only 0.82 (0.86 in your paper), PSNR was 23.53 (21.9 in your paper). I checked the generated images in the result folder while they were normal and the input images sucessfully translated into another expressions .

Here is my specific workflow:

  1. Download dataset and weight
  2. Pair face with landmark, and lead to a row of images [neutral_face_A, other_expression_B, other_landmark_B, neutral_landmark_A]
  3. Split the whole dataset, and get 1407 images for testset. (BTW, I tried different seeds to make testset, but the value of SSIM are all around 0.82)
  4. Run the test shell script with the default args you set before.
  5. Calculate SSIM and PSNR of each pairs of images and take the average.

Do I need to train the network by myself to reproduce the experiment, or I just did something wrong during the test process?

Ha0Tang commented 4 years ago

Which evaluation code did you use for calculating SSIM and PSNR?

sitadivon commented 4 years ago

I used API supported by scikit-image.

Ha0Tang commented 4 years ago

Can you try this lua evaluation code we used? Evaluation code written in different programming languages (e.g., Python, Matlab and Lua) will have slightly different results.

sitadivon commented 4 years ago

SSIM: 0.85847000799396 PSNR: 29.464491453809 Sharpness: 21.488623304232

I got right result by using your lua code. Thank you : )