vladmandic / face-api

FaceAPI: AI-powered Face Detection & Rotation Tracking, Face Description & Recognition, Age & Gender & Emotion Prediction for Browser and NodeJS using TensorFlow/JS
https://vladmandic.github.io/face-api/demo/webcam.html
MIT License
824 stars 149 forks source link

Pitch and yaw values aren't correct. #128

Closed TheSohaibAhmed closed 1 year ago

TheSohaibAhmed commented 1 year ago

Issue Description The yaw and pitch values are somewhat problematic (understandably). Somehow yaw remains in negative, and pitch changes as you shake your head (but don't actually tilt your head forwards or backwards).

Steps to Reproduce You can see this on the demo too: Keep your head straight; look to the left, and then to the right without pitching your head -- watch the pitch values change.

Expected Behavior Maybe if there was someway to make it more consistent?

**Environment On the demo itself

Additional Some thoughts re. yaw : Have you maybe considered measuring the ear-to-ear and ear-to-nose (X2) distances? Because if let's say the right ear to nose distance is shorter then we know we'd be looking to one side.

PS. Sorry for not a a lot of info on the system -- I noticed that the issue showed up in the demo and was consistent with my build and across browsers so I figured that it's maybe a universal problem? Let me know if I'm wrong or can help you fix this in anyway

vladmandic commented 1 year ago

if you're interested, feel free to contribute to @vladmandic/face-api :)

at the moment i'm maintaining it and fixing bugs as needed, but i'm putting all active development in the new library @vladmandic/human instead as face-api is architecturally quite old

more specifically, method i've implemented in face-api is older version that relies on simple angle calculation between vector lines:

i have a newer way to calculate angles using rotation matrix which i've implemented in @vladmandic/human: rotation matrix

TheSohaibAhmed commented 1 year ago

I'm not sure how to contribute to it, but I went ahead and fixed it (I think; fingers crossed!!). Here's codesandbox link: https://j6q0jb.csb.app/

Specifically, I went and selected specific points and created the right angles for it. It's still an approximate and I can refine it for sure, but it certainly looks more accurate =D

P.S. I'll look into how to officially contribute :)

vladmandic commented 1 year ago

its looks pretty good to me... instead of me copying and pasting the code, its best if you contribute directly:

vladmandic commented 1 year ago

thanks for the pr #130 - overall looks pretty good, i've left some minor comments before the merge.

TheSohaibAhmed commented 1 year ago

^Woot! Yep -- I'm glad this got sorted out, and I responded to those comments too! :D

vladmandic commented 1 year ago

pr is merged and version 1.7.6 is published
(in theory its a breaking change due to switch from radians to degrees, but i didn't think it deserved a major version bump)