Quest Pro Facial Tracking driving Pout incorrectly & Pupil Size

[BeagleDragon36]

Describe the bug?

A. When using the Quest Pro with eye and face tracking, for some reason, when you scrunch up your eye (close them and keep them shut forcefully) it drives the pout gesture.

B. There is also a bug where by default, the pupil diameter is -1 for the quest pro's face tracking.

To Reproduce

A: Load any avatar with face tracking, take a timed photo, and force your eyes shut. Result will show the pout gesture is active while not making a pouting face.

B: If you have "goober" eyes, its easiest seen through that. No pupils will show, and the eye manager will say pupil size is -1. I'm not sure if this has to do with it being a second eye manager, as I never checked the primary eye manager with my quest pro equipped yet.

Expected behavior

A: Pout gesture to not be driven, and eyes to be fully closed

B: Pupils to be either 0 or a normal, non-negative number

Screenshots

(for A, Balloonie has their eyes forced shut and its driving the pout gesture, even when theyre not making a pouting face)

(for B. This was taken after a slight adjustment to flux to compensate for the -1 and the massive - number supplied. Initially, the eyes were pure black)

Resonite Version Number

2024.5.22.1274

What Platforms does this occur on?

Windows

What headset if any do you use?

Quest Pro

Log Files

(Log provided has mods, but this issue has no correlation to said mods.) DESKTOP-R8JTGNI - 2024.5.22.1274 - 2024-05-22 22_47_44.log

Additional Context

When I first noticed this, I asked Balloonie and Scraner, who were in the world at the same time and both have Quest Pro's to test this, and both had the same things happen, with the pout being driven by eyes being forced shut, and pupil size being -1

Reporters

Beagle_36

[5H4D0W-X]

The pupil size being -1 is intentional to show that there is no data, here's the relevant section from the changelog:

- Eye tracking system is now aware of whether pupil tracking is supported or not
-- For eye tracking hardware that does not support pupil tracking (e.g. Quest Pro), EyeManager will automatically simulate it as normal
-- If you're accessing the raw data, the value will be less than 0 (-1)

[shiftyscales]

How is the avatar that is pouting set up, @BeagleDragon36? Could you check the AvatarExpression driver to ensure that it isn't misconfigured? Is the pout gesture misbehaving across all avatars? If so, could you please provide one for reference?

Without knowing how the avatar is configured, there is not much that can be immediately done with this issue.

[Frooxius]

Can you actually attach the AvatarRawMouthData and see if pouting is driven at all when you close your eyes?

The pupil size being -1 is normal yes - Quest Pro doesn't track it pupil size. The changelog specifically mentions this.

[BeagleDragon36]

So, I dont have any public avatars on hand, but this happens with every avatar that has a face tracking component and a Pout Gesture. adding the avatarrawmouthdata it does show the pout being driven

[Frooxius]

Are you sure you're not actually pouting when you do that?

The raw pout seems to be activated, meaning the headset itself is detecting it and sending us the data about it.

You could try covering your mouth and seeing if it still happens.

[BeagleDragon36]

I just tested it, with my mouth covered it still drive the pout gesture It also seems to be driving more than just pout too

[shiftyscales]

The requested testing has been done, @Frooxius.

[Renarikun]

I have a similar but opposite issue where some gestures like angry or pout will override the eye state and keep my eyes open when I have them shut or lowered. I can not frown and squint at the same time with the current tracking.

[Renarikun]

Follow-up: it appears most of my eye open/close tracking override was due to another issue on my avatar. Pout and angry still seem to have detection issues, but I'm not sure how much of this is Quest being dumb about it.