Pose estimation for an input image

[arunponnusamy]

Hello Patrik,

I am trying to use pose_estimation.cpp example code to detect yaw, pitch and roll for an input image. What is the size of the image from which you have taken the landmarks (Mat landmarks) ? Can I take image of any size and identify the landmark points using opencv/dlib and use it for pose estimation ?

What do you mean by image origin (500) ? How do we get approximate focal length (1800) ?

Any help is really appreciated. Thanks.

[patrikhuber]

Hi Arun,

I am afraid this was around 3 years ago (when we published the paper Fitting 3D Morphable Models using Local Features, ICIP 2015, so I don't remember these details from the top of my head. Your best bet it actually to read our paper. The pose_estimation.cpp example is actually directly taken from the following original paper by Xiong & De La Torre if I remember correctly: https://arxiv.org/abs/1405.0601. So you should find the exact details of how this works there.

Sorry I can't be of more help.

Actually, if you're interested in face head pose estimation, and you have already got landmark points, have a look at the eos library: https://github.com/patrikhuber/eos eos is under active development, and you can easily do head pose estimation and get yaw pitch and roll angles. It even comes with an example image plus landmarks and shows you how to retrieve the angles.

[arunponnusamy]

Thanks a lot for the reply Patrik. The code from the link referred in the paper has been taken down for some reason. http://www.humansensing.cs.cmu.edu/intraface

I built eos after dealing with some issues in upgrading cmake and gcc/g++. I was able to get YPR values from fit-model-simple example. I have a few questions.

1. Is pose estimation in eos based on SDM ? How is it different from superviseddescent implementation ?
2. How accurate the YPR angles predicted from eos and SDM? would you suggest one over the other ?

Thanks again for your time. Really appreciate your help.

[patrikhuber]

1. It's completely different. eos is landmarks & 3D model based, SDM is learning based (regression). Please see the respective papers for details. (the eos paper is linked here)

2. eos gives you pretty good head pose estimation, up to a few degrees or better I'd say. The fitting is really good, so it mainly depends on your landmarks. SDM is learning based so it depends on your training data (maybe Xiong et al. reported some accuracy values in one of their papers, you'd have to check).

As written above I would recommend eos, as is it actively developed and works very well for head pose estimation. (That's not to say you can't do it with SDM, I've got some pretty good landmark detection results with this SDM implementation too, even nowadays, but you'd have to put in some time probably to make HPE work well).

[patrikhuber]

I'm going to close this, feel free to post back if there's any follow-up.