[Suggestion] - Determine expression based on dialogue text

[PongMaster69]

Simply put the program to recognize the proper facial expression to make from the context of the sentence being input or output. For example words with a negative connotation would correlate to worried expressions or potentially even angry, and happier connotations would relate to happy or romantic facial expressions. I believe that it would be incredibly helpful for easily implementing new statements but also open up new possibilities, such as having Monika read things to you.

[ibm2431]

This is a subject tackled by sentiment analysis models like the (Revisited) Hourglass Model.

Academic work has also been done based on Plutchik's "Wheel of Emotions" by the Chinese Academy of Sciences in Beijing, in which they build a lexicon for emotional sentiment analysis: "EmoMix: Building an Emotion Lexicon for Compound Emotion Analysis".

In my considerations of how one might best design, develop, and implement a true emotional model for virtual agents (like Monika), I've been going back-and-forth on the value of developing a similar lexicon for "automagical translation of scripter's chosen word for how Monika feels to the true underlying emotion" versus enforcing strict adherence to limited vocabulary. In other words, if a scripter has to input Joy, or if the system would also "accept" words like Glee, Joyful, Glad, Rejoice, etc but internally translate them all to Joy anyway.

I don't think the existence of an emotional lexicon is necessary for the purpose of implementing base emotions (ie: strict adherence would be fine with developer education and guidance), but one would certainly have value in sensing overall emotional expression from freeform input by the player, or for use by Monika herself -- if such features are desired.

But if an emotional lexicon is something worth pursuing, emotional sentiment can be mapped to Facial Action Coding System to assist with or guess an appropriate facial expression for a line of dialogue by Monika.

[xXMidnight]

ooh

[ThePotatoGuy]

I'm confused as to what you are suggesting. Are you asking for recognizing the player's facial expressions?

Or is something regarding Monika's expressions. By virtue of writing dialogue, we have to pick the appropriate expression for Monika, so if there is a problem with a specific line, then the specific line should be mentioned either in an issue or in #7271

[PongMaster69]

It's about the having to manually pick the expression. What I'm proposing is that if someone were to make a program which would take text inputted and identify certain words to change Monika's expression. It doesn't need to be the fully free expression as aforementioned above, but rather something that detects an emotion from a sentence (I.E. words like "like", "confuse", "infuriate", "think", etc.) and swaps the sprite to match it (EX: change mouth set from face-mouth-smirk to face-mouth-pout; or eyes-normal to eyes closed; etc.).

This I believe would allow Monika to do things that she's talked about but never been able to do, I.E. reading with the player. Either by text uploaded or by maybe by pdf. This could also be beneficial to future implants of texts as one could implement a text, run the program and then run manual corrections, maybe allowing the process to be slightly easier. Mostly I just believe that this will be incredibly useful in general and help immensely with future changes.

[Booplicate]

This is more complex than you think. And even then we'd still have to manually review the final result.

[ibm2431]

SenticNet has released various versions of their sentiment analysis databases, with SenticNet 6 containing 200,000 concepts.

See their entry for Literature for an example.

Unfortunately the file download is riddled with XML bloat, coming in at 300 MB for a single XML file.

But while getting lost in the weeds of affective computing, I slimmed it down into a CSV and prettified spreadsheet: SenticNet 6.zip

The CSV contains all data originally part of SenticNet 6, including related concepts. The spreadsheet only contains final scores based on the Revisited Hourglass sentiment analysis model.

While they claim to have a high amount of accuracy for overall sentiment analysis, the scores for some concepts in this database seem questionable. But I've yet to perform any sentiment analysis with this database, so I can't offer any final judgment on its scoring.

[ThePotatoGuy]

So an NLP thing, I see. We are not planning to work on any NLP thing ourselves at this time. If a prototype that works with MAS is developed, then maybe we can revisit this.