Open qcj1206 opened 2 years ago
3D facial tracking is used to obtain 3D facial landmarks and head poses. Deca is an alternative option to get these parameters. In fact, any parametric 3D facial reconstruction method is considerable. Camera calibration is used to achieve better reconstruction.
Thanks,I still have some questions about the two preprocessed data. How can I get the idscale.mat and shoulder Point3d (I know this is calculated through LK optical flow, but how to get the key points of 3D)
id & scale parameters are results of 3D facial tracking. To get 3D shoulder points, we first detect 2D shoulder points using LK flow and reconstruct 3D shoulder points by assuming a billboard model where the depth is the mean 3D facial depth for each frame.
thank you again ! I trained the audio2feature and audio2headpose modules. Here are some questions I hope you can answer
How can I get a linear velocity term in Δ Pt audio2headpose
Whether tracked 3D in audio2feature is normalized? I used [0,1] normalization. It seems that there are some problems
In the article, proj in Fig. 2 stands for camera projection. How can I integrate the mouth headpose and eye information through this? My understanding is to convert 3D into 2D through the camera. So the rest of the landmark information adopts the original landmark of the video?
Whether HeadPose is relevant with contour(landmark) ?
- As for 3D face tracking, whether it is obtained from the original image(original video is 1920*1080)and whether it is affected by camera calibration, If I using deca to calculate the 3D key points for each frame,what is the different with two methods?
- What method is used to obtain GroundTruth of the HeadPose? ( I use OpenFace2.2.0 method to get the HeadPose GroundTruth )
Thank you again for publishing the code. Thank you very much!
Hello, I have two questions, can you help me. I use the method of OpenFace2.2.0 to get the HeadPose GroundTruth.
thank you so much for your marvelous work.
These two files are 3d facial tracking results (3D landmarks, head poses, etc.). Check the inference code and replace them with your tracking results works.
As I understand, the papers 1 and 2 (mentioned in paragraph 2 of sec 4.1) are used to extract the face landmarks. Did you just use the paper codes, or there are off-the-shelf tools in python?
[1] Automatic Acquisition of High-fidelity Facial Performances Using Monocular Videos. [2] Face2Face: Real-time Face Capture and Reenactment of RGB Videos.
As I understand, the papers 1 and 2 (mentioned in paragraph 2 of sec 4.1) are used to extract the face landmarks. Did you just use the paper codes, or there are off-the-shelf tools in python?
[1] Automatic Acquisition of High-fidelity Facial Performances Using Monocular Videos. [2] Face2Face: Real-time Face Capture and Reenactment of RGB Videos.
maybe you can see, foocker/LSP
As for 3D face tracking, whether it is obtained from the original image(original video is 1920*1080)and whether it is affected by camera calibration, If I using deca to calculate the 3D key points for each frame,what is the different with two methods?
What method is used to obtain GroundTruth of the HeadPose? ( I use OpenFace2.2.0 method to get the HeadPose GroundTruth )
Thank you again for publishing the code. Thank you very much!