search

YuanxunLu / LiveSpeechPortraits

Live Speech Portraits: Real-Time Photorealistic Talking-Head Animation (SIGGRAPH Asia 2021)
MIT License
1.16k stars 200 forks source link

Is it possible to use 2D landmark if not include head motion? #13

Closed forest520 closed 2 years ago

forest520 commented 2 years ago

Dear Dr. Lu: In the paper, sparse 3D landmark are used as an intermediate representation, and later project these 3D positions to the 2D image plane via pre-computed camera intrinsic parameters 𝐾 to get Conditional Feature Maps. I wonder whether it is ok to use 2D landmark directly if I don't need head motion as mentioned in #11 Thanks again for your answer!

YuanxunLu commented 2 years ago

If you don't care about head motion, 2D landmarks are also an option. The benefit of 3D landmarks is to disentangle the expression motion, head motion, and camera matrix. Directly using 2D landmarks entangles these parameters.