search

microsoft / SynthMoCap

Other
397 stars 21 forks source link

Head Pose and Intrinsics of SynthFace #8

Closed filby89 closed 1 month ago

filby89 commented 1 month ago

Hello, and thank you very much for sharing this dataset! I am trying to use the SynthFace data in order to render a 3D face with the camera intrinsics and extrinsics provided as well as the head pose. Some questions:

1) The head pose already includes the pose w.r.t. the image so camera extrinsics are not needed here, am I correct ? 2) If I want to render using the extrinsics/intrinsics and the head pose I believe I would also need the translation of the human face in the world. This is not given for SynthFace, but for SynthBody it was given. Do you have any plans on including this ?

Thanks !

friggog commented 1 month ago
  1. The head pose provided is in camera space, so you are correct that you don't need the camera extrinsics
  2. As we can't release the face model/dense face data the raw translation wouldn't be that useful as it would apply only for our model. One thing you could try is to take the sparse 2D landmarks, head pose and camera parameters along with your face model and solve for the 3D location given those inputs. Something like PnP can could do this ignoring the provided head pose, or you could use a general optimizer to find the best solution given all the information
filby89 commented 1 month ago

Hey @friggog thanks for your answer. You are right about 2, I think we could solve for the 3D location using the focal length and 2D landmarks, even taking the pose into account.