New Feature request - Facial emotion recognition

[YaronHershkovitz]

Hi, First of all, great job!!! & thanks a lot. I would like to use an emotion recognition ( happy / sad ...) ability in my project. I've detected the face already with the help of your example.
Do you have an example ( or a hint :) ) for implementing emotion detection? Is there ability for fiducial points recognition, ArUco markers?
Thanks...

[justadudewhohacks]

Hm I am not sure if OpenCV is able to do emotion detection, but correct me if I am wrong.

[YaronHershkovitz]

Thanks for your quick reply. You are right, Not out of the box. I thought about using fiducial points, but what I actually intended to do was finding the face landmark as you call it in your face-recognition.js package and compare the shapes of the mouth for example with a pre-defined shape for emotion recognition. so I guess i'll use your face-recognition.js for that and opencv4nodejs for the shape comparing & other stuff. Can I get the mouth points, eyes points... from your package? Do you think it's a good approach? Thanks.

[justadudewhohacks]

Yes you can get the 68 point face landmarks with face-recognition.js and you can use the package with opencv4nodejs. Using face landmarks for recognizing face expressions would also be my first guess.

[nicholasc]

@YaronHershkovitz You can use the FER2013 dataset to the train your model. You would only need to write a script that generates landmarks for each face and then use landmarks as features to predict emotions.

https://www.kaggle.com/c/challenges-in-representation-learning-facial-expression-recognition-challenge/data

There are features that can be calculated from landmarks that would give you more accuracy. For example, Affectiva uses the following features to train their model:

• Brow raise
• Brow furrow
• Nose wrinkle
• Lip corner depressor
• Chin raise
• Lip pucker
• Lip press
• Mouth open
• Lip suck
• Smile

[YaronHershkovitz]

@nicholasc thank you very much. I'll explore it.

[nicholasc]

Closing since an answer has been provided and the proposed feature is out of the scope of this package.