How to get 3D position of Head?

[prometuse]

Hello, I am new to facetracker. I am recently doing a project to do gaze estimation, and I am using facetracker. I need the infomation of 3D head position w.r.t the camera, is Facetracker offering that? Thank you very much !

[jovan-fernandes]

I think that the information about Face Pitch, Face Yaw and Face Roll are on the vector "FaceTracker ->_clm._pglbl". The first one ( at(1,0) ) is the pitch, the second is the yaw ( at(2,0) ) and the last one is the Roll ( at(3,0) ). Other information are also in this vector, perhaps kyle could answer what means the content at the position 4 and 5 of this vector.

[prometuse]

Thank you very much! I tried to print out content of (4,0) and (5,0). They are probably the coordinate of head on the 2d image plane. What I need is the 3D position including the depth info of face w.r.t camera. Hope somebody could give me a hint?

[jovan-fernandes]

Well, to get the head position in an opencv type vector you could do something like this:

const Mat& mean = faceTracker->_clm._pdm._M;
const Mat& variation = faceTracker->_clm._pdm._V;
const Mat& weights = faceTracker->_clm._plocal;
vector<Point3f> face3DPoints = mean + variation * weights;

With this code you get the facial points in 3d. That is what you want?

[prometuse]

@jovanoasis Thanks very much again! The code you offered is very helpful, but it seems to be the coordinate of a 3D distribution model relative to the head because the coord didn`t change when I move my head relative to the camera. So what I actually want is the depth info of head, usually obtained by a kinect.

[kylemcdonald]

for more examples of using data from inside FaceTracker, check out https://github.com/kylemcdonald/ofxFaceTracker/blob/master/src/ofxFaceTracker.cpp

to get 3d position, you have a projective version here:

https://github.com/kylemcdonald/ofxFaceTracker/blob/master/src/ofxFaceTracker.cpp#L247-L250 https://github.com/kylemcdonald/ofxFaceTracker/blob/master/src/ofxFaceTracker.cpp#L253-L256

these tell you 2d position and scale. from there you can convert to 3d position if you have a model of the camera intrinsics.