search

TadasBaltrusaitis / OpenFace

OpenFace – a state-of-the art tool intended for facial landmark detection, head pose estimation, facial action unit recognition, and eye-gaze estimation.
Other
6.95k stars 1.85k forks source link

how much faster is openFace face detector than dlib frontal_face_detector on mobile device? #749

Open bchen3 opened 5 years ago

bchen3 commented 5 years ago

Hi, i'm currently using dlib in my ios project for both face detection and landmarks finding. Yet the face detection process is too long to be used for live frame. Before switching to openFace, I'd like to ask, how much faster can openFace detector run than dlib in face detecting when used on mobile device (saying iphone6)? Thanks.

TadasBaltrusaitis commented 5 years ago

OpenFace uses a re-implementation of the MTCNN face detector internally (https://github.com/ipazc/mtcnn, https://kpzhang93.github.io/MTCNN_face_detection_alignment/). I have not compared the speeds of the dlib and MTCNN detectors in great detail (although on my machine the MTCNN ones is slightly faster).

I really don't know how well that would translate to the IOS case though.

bchen3 commented 5 years ago

Thanks for your reply. Another question, what image-preprocessing suggestions would you make on face detection for better results, since I found out that certain pre-processing touches on any given images increase the accuracy?

TadasBaltrusaitis commented 5 years ago

While a number of approaches do indeed get better with image pre-processing, I don't think MTCNN is very sensitive to them (although I did not spend time investigating it in depth).

bchen3 commented 5 years ago

Thanks for your reply. Another question, would you suggest use any tracker, take dlib correlation_tracker for example, for face alignment in real time face detection? I did test dlib correlation_tracker() to track face in live video frames, yet foud out it was not as good as I supposed since it lost face very easily when face moved around quickly. Any idea why this happened? Need your option! Thanks!

------------------ 原始邮件 ------------------ 发件人: "Tadas Baltrusaitis"notifications@github.com; 发送时间: 2019年7月24日(星期三) 凌晨0:33 收件人: "TadasBaltrusaitis/OpenFace"OpenFace@noreply.github.com; 抄送: "陈波"1132756202@qq.com;"Author"author@noreply.github.com; 主题: Re: [TadasBaltrusaitis/OpenFace] how much faster is openFace facedetector than dlib frontal_face_detector on mobile device? (#749)

While a number of approaches do indeed get better with image pre-processing, I don't think MTCNN is very sensitive to them (although I did not spend time investigating it in depth).

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

TadasBaltrusaitis commented 5 years ago

OpenFace uses a slight amount of tracking internally, using some basic correlation tracking for initialization and initializing the current detection from previous detection.

While tracking can often be faster, you already noticed that it often leads to drift, which you then need to correct for using some form of an external method. OpenFace does that by checking if landmark detection succeeded (using a very basic CNN)