search

TadasBaltrusaitis / OpenFace

OpenFace – a state-of-the art tool intended for facial landmark detection, head pose estimation, facial action unit recognition, and eye-gaze estimation.
Other
6.99k stars 1.86k forks source link

Real-time detection with a webcam #968

Open summer-ops opened 3 years ago

summer-ops commented 3 years ago

Sorry,my English is not good. 1.The existing Matlab scripts are all input video files.How should I modify the code so that I can open the webcam for real-time detection? 2.Do you have any information about OpenFace of gaze estimation and head posture estimation? Thank you!

TadasBaltrusaitis commented 3 years ago

1.The existing Matlab scripts are all input video files.How should I modify the code so that I can open the webcam for real-time detection?

For working on webcam data I suggest you use the C++ code. as Matlab implementation is not fast enough to deal with real time video, and I'm not sure if Matlab has a nice webcam interface

2.Do you have any information about OpenFace of gaze estimation and head posture estimation?

You can find the description of OpenFace in the following papers: OpenFace - https://www.cl.cam.ac.uk/research/rainbow/projects/openface/wacv2016.pdf OpenFace 2.0 http://multicomp.cs.cmu.edu/wp-content/uploads/2018/11/OpenFace.pdf

BaiqiangGit commented 3 years ago

@summer-ops, besides, the following is an example command to run with webcam with the exe:

./bin/FaceLandmarkVid -device 0

simonmssu commented 3 years ago

I am interested only in the head position tracking (positional and orientation data) in real time using the webcam. Running Feature Extraction.exe -d device 0 and FaceLandmarkVid.exe -d device 0, I am getting anywhere from 5 fps (actively moving my head) to 30 fps (no head showing in the webcam view) using a cheap logitech webcam. Is there any hardware (GPU/CPU/memory) or software (just process head pose and not doing gaze estimation or parallelization) optimization that will maintain a real-time tracking of at least 25 fps? thanks....