search

TadasBaltrusaitis / OpenFace

OpenFace – a state-of-the art tool intended for facial landmark detection, head pose estimation, facial action unit recognition, and eye-gaze estimation.
Other
6.98k stars 1.85k forks source link

Initialize CLNF Models with predefined head pose #223

Closed CMoench closed 7 years ago

CMoench commented 7 years ago

Dear Tadas, I'm currently working on a small study to evaluate the AU intensity estimation in videos recorded from different angles. I will compare time series of AUs from a frontal reference camera with these of other cameras recording the face from different angles. Problem is that the face detection and also the landmark detection algorithm detects nothing in most cases, if e.G. the horizontal angle is above ~ 40°. If the CLNF model is already active and I turn my head above 40° the tracking works. So I'm wondering if I can initialize the CLNF model with the head pose (location and orientation) because I know these and I can use these information. Can you tell me a way how to achieve that? ( If you know a way to tweak the algorithm to detect faces/landmarks recorded from greater angles / from the side automatically, this would be very useful as well ;-) ) Best regards Christoph

TadasBaltrusaitis commented 7 years ago

Hi Christoph,

Yes you can, and there's some code that already does that. If you look at LandmarkDetectorFunc.cpp and specifically DetectLandmarksInImage function if the params.multi_view is set to true it will try detecting landmarks at different orientations set in the rotation_hypotheses variable. You can change that variable to correspond to the rotations you are expecting in the image, either by leaving only one rotation or adding more hypotheses.

Hope this helps.

Thanks, Tadas

CMoench commented 7 years ago

Awesome!! It works! Thanks a lot!