search

patrikhuber / superviseddescent

C++11 implementation of the supervised descent optimisation method
http://patrikhuber.github.io/superviseddescent/
Apache License 2.0
402 stars 188 forks source link

Way to detect whether the detected landmarks are actual landmarks on a face #50

Closed fleuguil closed 7 years ago

fleuguil commented 7 years ago

Hello Patrik,

first I want to thank you for making available such a great piece of software.

I did a few tests on multiple images and the position of the found landmarks is usually quite good. I have however encountered a problem when using the OpenCV face detector.

Sometimes, the OpenCV V&J face detector detects things that aren't actually faces. The supervised descent code still runs and things that aren't landmarks on a face are detected.

My question is :

By looking at the way the optimization goes after each steps or some other method, is there a way to tell that the function is not being optimized properly? In the case of a face, that could be used to decide whether the landmarks were found successfully of not. What I mean is when doing the supervised descent, can we look at the update_step or some other method and tell if things are going the right way or not? The problem with faces is as you mentioned that we have no ground truth to compare our results with because every face is different. Is there a way to tell that we are close or very far from the best solution?

Sorry, I am just an engineer and not a computer vision scientist but I would really appreciate if you could give me some useful pointers.

Regards, Guillaume

patrikhuber commented 7 years ago

Hi Guillaume,

Thank you very much!

The SDM does not offer a direct way to find out whether it's actually converging on the actual face. It's just a set of regressors that give you a shape update step. However, many methods learn a confidence classifier (or regressor) along with the actual landmark detector, to provide exactly that, a measure of how confident it thinks it is. You can learn that in exactly the same way as the actual landmark regressors, for example just with a confidence value as the regressor target value. There's also more complex methods and you'll have to see what works for you. Some people have asked this question before as well so if you browse through the GitHub issues (closed ones too) you should find some more pointers.

Regards, Patrik

fleuguil commented 7 years ago

Hi Patrik,

Thank you very much for the insights.

After looking at the other questions, I think I should try to train a linear classifier at the same time I train the supervised descent using the HOG features. Again I am not a computer vision scientist and if it just work reasonably well that might be good enough for me. You've pointed me in the right direction, at least I know where to look now.

Thanks again. Guillaume

patrikhuber commented 7 years ago

Yes, pretty much. I personally don't have much experience with that but I guess it'll depend on the training data how well it will work, and maybe on what measurement you choose for the target label. It's also possible that a more elaborate classifier than linear might be necessary to get really good results. Not sure ;-) But you should at least get an initial working version using this simple approach.

Good luck!

I'll close this but feel free to follow-up / report back.