input ref output result

[ak9250]

How do you test for the input ref output result shown in the readme, can you test on your own images?

[ak9250]

I am referring to these results https://github.com/zhengziqiang/ReshapeGAN#results

[zhengziqiang]

I have reorganized the codes, and you can generate one single image or multiple images at once.

[ak9250]

thanks, there is no google drive like for celeba_images.zip in the Readme

[zhengziqiang]

@ak9250 I have uploaded the two files with the codes, you could download these with the added testing codes.

[ak9250]

@zhengziqiang sorry, I see it now, also I tried it with my own images and the result demo_celeba.jpg is cut off and weird

[ak9250]

same weird results with provided celeba test images

[zhengziqiang]

In our paper, we feed the model with the RGB image and geometric guidance (landmark, skeleton or even key point annotation). And for the weird results you mentioned, we have explained in our paper, if the distance between two face images is too large, the output would not be satisfactory. For example, if the input image is from a woman and the reference image is from a man, the generated image would be somehow weird. And you can run the test_multiple.py to generate more images and visualize all the results.

[zhengziqiang]

@ak9250 For the first case you mentioned, you should feed the model with real RGB image and geometric guidance.

[ak9250]

@zhengziqiang can you share the script to get the geometric guidance for the test images?

[zhengziqiang]

You can also try other combinations by changing he path of input image or reference image. In theory, you could generate 1000*1000 images using 1000 images.

[ak9250]

@zhengziqiang ok, can you please add the code on how you process the images to get the celeba_images test images with image and geometric guidance

[zhengziqiang]

Ok, I will upload the code to extract the geometric guidance as possible as soon. Any problem for the evaluation?

[ak9250]

@zhengziqiang I just ran test multiple and here is the first result

[zhengziqiang]

@ak9250 I will also upload the code to merge all the synthesized images to a larger image, which is similar to the shown image in our paper. As you can see, the 1st, 3rd and 5th synthesized images have better reshaping performance than the 2nd and 4th outputs. The reason is that the geometric distance is too large between the input image and reference image. Our model fails to handle this task for the lack of prior knowledge. Even in the training stage, we could only obtain one image from one identity.

[zhengziqiang]

To make it easier to show the reshaped results, I will upload the merge_results.py this evening.

[ak9250]

@zhengziqiang thanks are you getting the 68 facial landmarks through the script, anything like dlib should work?

[zhengziqiang]

Yeah, I just used the dlib to obtain the facial landmark.

[zhengziqiang]

@ak9250 I used this code: https://github.com/davisking/dlib

[ak9250]

@zhengziqiang ok, also i suggest setting this up in google colab as most of these come preinstalled in there as well as scipy==1.2.0 is needed so you dont get imread or imsave errors

[zhengziqiang]

@ak9250 Thank you for your suggestions, I have already uploaded the merge_results.py to make it easier to show the synthesized images.

[ak9250]

@zhengziqiang thanks i tried out the merge results

I would like to try this with my own images but still setting up dlib for batch processing and combining, can you share your script as it would be faster?

[zhengziqiang]

@ak9250 Ok, I will share my own codes to extract the facial landmarks for batch processing. But now it is too late, I will upload the codes tomorrow.

[zhengziqiang]

@ak9250 If you have interesting on GAN, you could see my another article: EncryptGAN: Image Steganography with Domain Transform, to our knowledge, which is the first asymmetric image encryption system based on GAN. The paper could be found at https://arxiv.org/abs/1905.11582 and the code will be released at https://github.com/zhengziqiang/EncryptGAN

[ak9250]

@zhengziqiang thanks, I ran extract_landmark.py and the celeba_land folder is empty

[ak9250]

@zhengziqiang I was able to use another method and save the image does it need to be exactly in this format?

[ak9250]

I was able to get it into that sort of format but results are weird still

[zhengziqiang]

@ak9250 You should feed the model the center cropped image. And the right of the input image should be the facial landmark image like this.

[ak9250]

@zhengziqiang yes, I tried using extractor_image.py and it gives empty output, nothing in directory

[zhengziqiang]

@ak9250 You can try it now. The file list is empty while using glob.glob(data_path).

[ak9250]

@zhengziqiang tried it now and it is working but still getting weird results

[ak9250]

@zhengziqiang any idea why the weird results are happening?

[zhengziqiang]

@ak9250 You should normalize the both two images similar the testing data provided. Why do you use the input-reference images, which has different sizes of people? You only cropped and spatially normalized the reference image, and you don't implement the same operation to the input image. In our experiments, we have performed the same operation to the all the images, which could be found in facial_normalized.py. The two images have obviously different sizes (the facial landmarks)!

[ak9250]

@zhengziqiang yes, I did use facial_normalized.py which gave me the merged and normalized results, I used the merged results in testing, will look into the normalized results