search

TadasBaltrusaitis / OpenFace

OpenFace – a state-of-the art tool intended for facial landmark detection, head pose estimation, facial action unit recognition, and eye-gaze estimation.
Other
6.96k stars 1.85k forks source link

Some questions #51

Closed cyberjunk closed 7 years ago

cyberjunk commented 8 years ago

Hey Tadas,

very impressive work! I'm currently writing my thesis and I'm tending to make use of your library in it. I've got like a million questions after reading through your papers and code for days, but I'll try to limit them to a few generic ones here first :-)

1) Would you like to share with us what your next steps/improvements on OpenFace might look like?

2) Do you plan to add something like 'Gender' or 'Age' estimation at some point?

3) Are you looking for contributions? Which are the aspects of OpenFace that you would like to see improved, but don't have enough time to do so by yourself?

4) The version 0.2 suggests the lib is at an early development stage. Do you plan to do any major API-breaking rewriting/refactoring within the next couple of months?

5) Just some feedback: In my tests, the AU45 ('Blink') classifier detects closed eyes well, but unfortunately also shows a lot of false-positives (e.g. when moving/rotating the head around with eyes open). However, at least the AU45 regressor value nicely peaks for closed eyes and the face position and orientation estimations are just awesome anyways!

Kind regards, Clint

TadasBaltrusaitis commented 8 years ago

Hey Clint,

Thanks for taking an interest in my work. To answer some of your questions:

  1. Depends, I have a number of projects on this in parallel trying to improve all aspects of the model - landmark tracking, facial action unit estimation and gaze estimation. But as they are research projects not all of them will end up in OpenFace.
  2. I was thinking of that for some time and was planning on adding it at some point, but not in the very near future.
  3. Very much so. The gender and age estimation would be one of them. Another feature that I wanted to add for some time was head nod and head shake detection. There is also lots of potential for optimization using GPUs that I never got around to doing. I also wanted to add a better face detector as that affects the whole pipeline, dlib is quite good but it can't handle non frontal faces very well.
  4. Hopefully not, the API has undergone a reasonable amount of changes before, but hopefully it will stay reasonably stable from now on.
  5. Thanks for the feedback. Sometimes the AU detection is indeed noisy. And as you noticed regression is often more accurate as the peaks correspond better with activations than the detection values. I have some ideas to improve AU detection, but very much at the prototype stage, so watch this space.

Thanks, Tadas