auto set up lip tracking heurestics issue/tweak

[InsaneGrox]

Is your tweak request related to a problem? Please describe.

Several, all of them having to do with my avatar not correctly automatically setting up with lip tracking, specifically eye_frown shape keys being detected as mouth frowing shape keys, duplicate keys being detected causing overdriving, some keys straight up not being detected.

Describe what would you like tweaked

Eye_Frown should not be detected as mouth frowning, also if tongue_out exists, blep should be ignored. and finally, eye tracking needs some heurestics imo, widening, pupil dilation, squeezing, and frowning are all not being automatically set up for the eye manager. there's also a few shapes that aren't being detected.

Describe alternatives you've considered

N/A

Additional context

This issue was created at the request of Shifty in the discord, screenshots from original discord post below, I can give a sample model, but I will not attach it to this issue due to privacy reasons. https://cdn.discordapp.com/attachments/439422161657135106/820443214343176202/2021-03-13_17.57.53.jpg https://cdn.discordapp.com/attachments/439422161657135106/820443219007111199/2021-03-13_17.58.22.jpg https://cdn.discordapp.com/attachments/439422161657135106/820443224912691290/2021-03-13_18.00.15.jpg https://cdn.discordapp.com/attachments/439422161657135106/820443222701899786/2021-03-13_17.58.28.jpg

[Frooxius]

Eye_Frown is intentional. Currently the eye tracking doesn't actually detect frowning, so it looks more natural when it's detected from the mouth. Same with smiling.

It's a bit tricky situation. One thing I could do is have EyeManager take data from the Mouth tracking as well and use either, that could be a good solution.

I'll look into setting up heuristics for others, but it's a bit tricky, since there's a lot of naming conventions and things become ambigous, so the change of setting things up incorrectly is much higher.

E.g. eye widening can be mapped to a lot of different things, but also to some wrong things. I'll see about adding some that might be safe, but I'd need more examples.

The biggest problem are the pupils though. Usually these require some manual adjustment, because the way the dilation is done depends a lot on the avatar. On some I have to overdrive the blendshape from -0.5 to 1.0 or even 1.5 for it to work well. The eye tracking gives you the pupil size in millimeters, and without a convention on how to map it on different types of avatars it's hard to do heuristics for it that wouldn't mess things up.

[Frooxius]

Can you check if the "blep" and "tongue out" are both actually being setup? The heuristics have a mechanism that should prevent this from happening, so I'm not sure what's going on there.

[InsaneGrox]

Can you check if the "blep" and "tongue out" are both actually being setup? The heuristics have a mechanism that should prevent this from happening, so I'm not sure what's going on there.

It no longer sets up both... though lip raise upper and lower are undetected and should probably be tweaked...

[InsaneGrox]

also jaw open isnt being detected either for some reason?

[3x1t-5tyl3]

Yep I have issues aswell here. "CheekSuck" get's assigned to CheekSuckRight for some reason? JawHorizontal -> JawRight TongueOut -> TongueOut (this is correct) But then one line lower.. TongueExtend to TongueOut again?

https://pastebin.com/UvnsDmpR

[Frooxius]

@3x1t-5tyl3 It's getting confused by your naming convention, I'd strongly recommend sticking to the recommended standard naming for those blendshapes from the HTC SR Anipal SDK, rather than coming up with your own, otherwise you're relying on quirks of the heuristics, which won't always pick the right things: https://docs.google.com/spreadsheets/d/1EyD6JBV8sfeFEYvY2YNrWd9pfvK5nGXZOCn-Z5XMX4g/edit#gid=0

I'm adding some improvements to the heuristics that should fix it in your cases, but even with that, you should really just stick to the naming convention we recommend, otherwise it ends up picking up stuff you might not want.