search

VinAIResearch / CPM

💄 Lipstick ain't enough: Beyond Color-Matching for In-the-Wild Makeup Transfer (CVPR 2021)
https://thaoshibe.github.io/CPM
Apache License 2.0
367 stars 58 forks source link

Unsatisfied results tested on customized images #12

Closed BIGJUN777 closed 2 years ago

BIGJUN777 commented 2 years ago

Hello

I got some non-ideal results when testing the model on other images.

  1. As shown in the following image, there are some distinct boundaries on the resulting image. Any idea on this?

    image

    When diving into the code, I found you applied Face Detection before PRNet (See here). Because the image I input has been cropped, I commented those codes related to Face Detection in process() function (See below). However, the result turned out to be worse. It seems this Face Detection is necessary. Would you provide me with more explanation on this?

    image

    image

  2. When testing on my device (Tesla V100), it took several seconds to process an image, which is really time-consuming. Is this normal? If so, why will it be such time-consuming (it seems the model is not really complex)?

Looking forward to your reply. Thanks.

thaoshibe commented 2 years ago

Thanks for your interest in our work. I'm happy to help!

  1. About the boundaries Currently, we employed the pre-trained model of PRNet. As PRNet is not a 3D full-face model, it is expected that there will be artifacts in boundaries (Fig. 1). For better results, I suggest: 1.1. Using image blending technique 1.2. Employ other 3D full-face models. You can easily use any other 3D full-face models to replace PRNet in our framework.

Fig. 1. Reconstructed face from PRNet

  1. About face detection Using face detection and cropped face ensure the quality of the PRNet (See more Issues about that here). So yes, you can comment out those codes, but it will affect the performance of the 3D face model, subsequently cause unpleasant results (as you experienced).

  2. Testing speed I tested on a single RTX 1080Ti, running time ~ 1-2 seconds per image. The slowest part of the code is the 3D parts L34-36. I'd recommend you to check whether the code is currently run on GPU or not (As maybe tensorflow-gpu is not working). After that, further details to speed up the 3D model parts can be found on PRNet's repo.

Feel free to ask if you have other questions ( ❛ᴗ❛ )

BIGJUN777 commented 2 years ago

Thank you for your detailed as well as useful reply.

For the testing speed, as you mentioned, there is something wrong with the TensorFlow-GPU in my env. Will fix it later.