Research approach concerning emotions of people with PIMD using physiological parameters and facial expressions

[ghost]

Dear Maurya,

For a research project, I want to take a deeper focus on the combination of physiological parameters and facial expression to analyse the emotional expression of people with profound intellectual and multiple disabilities (PIMD). During my search, I came across your ideas and codes and, maybe, they are suitable to my approach.

Shorts explanation of the target group: First, each person with profound intellectual and multiple disabilities is very individual concerning her/his competencies and impairments. However, there are some characteristics that apply to a large number affected persons:

• profound intellectual disability (IQ < 20) combined with other disabilities (e.g., motor impairment, sensorial disabilities (hearing or visual impairment))
• communication: usually no verbal language
• usually no understanding of symbols
• maybe no use of common behaviour signals (e.g., different showing of facial expression in comparison to people without disabilities) -> for example, “smiling” is not always a signal for happiness

So, the problem is that these target group cannot tell us directly how they feel. Therefore, I created the following plan and maybe you can say me, if this is possible (with your software):

1. I want to trigger special emotional situations for the person with disabilities based on the information of her/his parents and caregivers.
2. These situations should be recorded with a focus on the face.
3. Afterwards, the personal facial expression of an emotion can be extracted -> Several Pictures of several situations that show the same facial expression, which stand for one emotion. The same procedure for other emotions.
4. The last step includes a field trail, in which these emotions should be recognisable using machine processing/software in daily life.

Do you think that I can train your software with the pictures that I get from the special emotional situations and use this trained software to recognize in the specific facial expression in a totally new recording? So, is it possible that the software can detect the shown facial expression (which will stand in the final analysis for an emotion) in a video? Moreover, is it possible to get some further details (like keypoints etc.) of the shown facial expression to use them in a further analysis?

To sum up, an answer would be really helpful. Thanks!

You can also contact me directly: t-s-m@outlook.de

Best regards

[MauryaRitesh]

Well, sorry for the late response.

When I created this project(Video for it is available here: https://youtu.be/Dqa-3N8VZbw) with the similar idea for understanding and analysing the emotions of the disabled according to their faces. However, this is not the most Accurate, yet I've been planning for making it work more accurately. For example, if we can feed the image of their face as well as their current body activity, then we could achieve a greater accuracy. BTW, This Project can be used regarding your needs. So feel free to Clone and edit it for meeting the most of your needs. If you wanna support this project and get a Higher Accuracy and make it work in the Real Life, please support me on Patreon(https://www.patreon.com/mauryaritesh) as i'll be working on this project.