Emotion recognition using openface !!

[Mabrar92]

I want to recognize six basic emotions using openface by utlizing the AU's output of the openface's. I am a beginner in this field (but an excellent programmer) can you please guide in right direction to do that. how can i extend the present code to do that ? any sort of help would be very useful. Thank You.

[TadasBaltrusaitis]

Hi, emotion recognition is a challenging problem and the perception of emotion will be heavily context dependent.

One easy, but not very accurate way to do it would be to use manual rules mapping AU outputs from OpenFace to emotions based on the EMFACS table here - https://en.wikipedia.org/wiki/Facial_Action_Coding_System, this, however, does not capture a lot of subtlety in expression intensity and dynamics.

Another option would be to use a dataset of basic emotions and to train a machine learning predictor based on OpenFace outputs (e.g. AUs) to predict the emotions.

[axatgupta]

Hello, I have started using the openface library. I want to detect basic 6 emotions using openface. I did refer to the FACS rules, but how should I go about extracting the AU's and what is to be done after that?

[TadasBaltrusaitis]

You can either use the GUI in Windows, which is fairly self explanatory or find instructions of how to extract AUs from command line in the following wiki link:

• https://github.com/TadasBaltrusaitis/OpenFace/wiki/Command-line-arguments

[gilinacio]

I searched everywhere and cannot find a way to convert the Action Units output from the CSV to the basic emotions as outlined in the facs manual. Is there any tool or script for that? I leave here some basic table to identify emotions from AU's. I can't seem to find an algorithm that does this automatically.

OpenFace seems very good and giving us raw data but we need a way to auto convert it. That is useful when you are dealing with a lot of data because you cannot do it individually. I was testing openface with about a hundred "10 second videos" and when I converted the data to the elan format it was quite easy to see an aproximate emotional response by looking into the AU but it gets very time consuming.

How do you analyse the data that openface gives us?

[TadasBaltrusaitis]

I made an explicit choice in OpenFace to recognize facial expressions (action units such as smile, brow raise, etc) and behavior descriptors such as head pose and eye gaze instead of emotions (things like happy/sad/etc.). The reason for this is that the former are objective measures what the face is doing, while the latter are much more subjective and open to interpretation + dependent on culture/context/age/gender. I find it helpful to think about facial expressions as the signal, and emotions as the message. While there are commercial tools that predict "emotion" out there, they are often exaggerating their capabilities, as recognizing internal emotion of someone without additional context is almost impossible.

While there are "rules" for converting facial expressions to a set of "basic emotions" they are just rough guidelines and not very accurate due to the subjectivity and ambiguity of the task. Before you go down that route I would reconsider what exactly you are trying to measure.

Instead of measuring how "happy" someone is, you can instead measure how much they smile, or look at things like lowering of brows which are more often associated with negative feelings (although not always).

Looking at expressions + head pose + eye gaze, allows you to get at a more raw signal.

[43zxj]

I searched everywhere and cannot find a way to convert the Action Units output from the CSV to the basic emotions as outlined in the facs manual. Is there any tool or script for that? I leave here some basic table to identify emotions from AU's. I can't seem to find an algorithm that does this automatically.

OpenFace seems very good and giving us raw data but we need a way to auto convert it. That is useful when you are dealing with a lot of data because you cannot do it individually. I was testing openface with about a hundred "10 second videos" and when I converted the data to the elan format it was quite easy to see an aproximate emotional response by looking into the AU but it gets very time consuming.

How do you analyse the data that openface gives us?

Hello, my next research is going to use the correspondence between AU and emotion. I would like to use the content of this image as a reference. Could you please tell me the source of this image?

[FereshtehFeiz]

I want to know the possibility of mapping the AUs to the facial emotions? I am doing research to know how have the facial emotions feature with OpenFace

[Xylia93]

The following paper provided action unit descriptions and the combination of each emotion, see Ko, H., Kim, K., Bae, M., Seo, M. G., Nam, G., Park, S., ... & Lee, J. Y. (2021). Changes in Computer-Analyzed Facial Expressions with Age. Sensors, 21(14), 4858. However, I want to know how can we use these AUs to represent the specific facial emotion.

[FereshtehFeiz]

How I can run a pre trained model to predict the emotions per each frame through the live webcam camera? and I want to show its output on the GUI. If anyone knows help me

Thanks

[TaniaZaheer]

I want to recognize six basic emotions using openface by utlizing the AU's output of the openface's. I am a beginner in this field (but an excellent programmer) can you please guide in right direction to do that. how can i extend the present code to do that ? any sort of help would be very useful. Thank You.

Hello, I am a beginner in this field too and if you have done this work, would you please share the source code with me too? it will really really help me a lot. Thanks