search

joelmoniz / DepthNets

Code for "Unsupervised Depth Estimation, 3D Face Rotation and Replacement", NeurIPS 2018
MIT License
127 stars 31 forks source link

For custom dataset, Which keypoint model should I use? #15

Open kdh4672 opened 3 years ago

kdh4672 commented 3 years ago

I finished the example that makes a man's face be warped to Obama's face.

As long as I know, If I want to warp custom face to another, I should prepare 2d keypoints for source and target face.

Can DepthNet model output 2dkeypoints?

If it is not, can you recommend good 2dkeypoints model?

Thank you!

christopher-beckham commented 3 years ago

Hi,

Sorry for the enormously long delay in response, I am sure by now my response is most likely irrelevant because you've either found the answer or moved onto something else. No DepthNet by itself cannot regress 2D keypoints. Internally, when we needed to extract 2D keypoints from a face we used @SinaHonari 's recombinator network, though if I remember correctly the implementation was in Theano (which by today's standards is ancient). You should use whatever off-the-shelf keypoint extractor you can find online (there should be many).

If the keypoint extractor ends up extracting 68 keypoints, you could use this diagram to help you figure out what two keypoint coordinates you need to remove in order to make it 66 (it's a bit of a hack but it should suffice):

https://github.com/tarunk04/P1_Facial_Keypoints/blob/master/images/landmarks_numbered.jpg?raw=true