Yellow-Dog-Man / Resonite-Issues

Issue repository for Resonite.
https://resonite.com
120 stars 2 forks source link

Add VRCFT Unified Expression names to Face Tracking Heuristics #2292

Open JackieCoyote opened 2 weeks ago

JackieCoyote commented 2 weeks ago

Is your feature request related to a problem? Please describe.

VR Avatar creators are starting to adhere to their own standard, meaning alot of avatars that are coming out are using Unified Expressions leading to a poor new user expierence for those with facetracking avatars cause our auto heuristics dosen't look for those names leading to manual setup which with some new users who are alone or have social anxiety. which is yet another hurdle for them.

Describe the solution you'd like

Add the naming convention for Unified expressions to our face tracking heuristics as closely as we can for ease of use for future avatars that continue to support the naming scheme. Linked below is a chart that shows unified names vs SRanipal, ARKit and Meta Move for ease of name comparsion.

https://docs.vrcft.io/docs/tutorial-avatars/tutorial-avatars-extras/compatibility/overview

Describe alternatives you've considered

Nothing really, the only other option is we don't and the new avatars that come out with these blendshape names dont get auto detected and we manual setup everytime

Additional Context

No response

Requesters

JackieAlexender

PointerOffset commented 2 weeks ago

I'd be quite happy to see this as well. The current heuristics tend to get me most of the way to finished face tracking setup. However, I almost always have to go into the AvatarExpressionDriver component and manually adjust or setup new assignments. This can be really tedious given how many expressions a user has to dig through and without UI improvements added to enum selection.

One of Resonite's nicest conveniences is when face tracking "just works" and I'd encourage adding Unified Expressions to these heuristics in service of that.

kazu0617 commented 2 weeks ago

Add the naming convention for Unified expressions to our face tracking heuristics as closely as we can for ease of use for future avatars that continue to support the naming scheme. Linked below is a chart that shows unified names vs SRanipal, ARKit and Meta Move for ease of name comparsion.

https://docs.vrcft.io/docs/tutorial-avatars/tutorial-avatars-extras/compatibility/overview

One effective alternative would be to change the name of the shape key in the model data to something heuristically detectable. In fact, there are names for blendshapes that can be heuristically detected, and I have had avatars I have requested in the past match those names.

BlueCyro commented 2 weeks ago

Add the naming convention for Unified expressions to our face tracking heuristics as closely as we can for ease of use for future avatars that continue to support the naming scheme. Linked below is a chart that shows unified names vs SRanipal, ARKit and Meta Move for ease of name comparsion. https://docs.vrcft.io/docs/tutorial-avatars/tutorial-avatars-extras/compatibility/overview

One effective alternative would be to change the name of the shape key in the model data to something heuristically detectable. In fact, there are names for blendshapes that can be heuristically detected, and I have had avatars I have requested in the past match those names.

This requires that you go in and manually re-name several dozen blendshapes to detectable keywords, which is what the core of this issue is asking to solve - so that people don't need to modify otherwise fully-featured face tracking avatars to yet another standard.

shiftyscales commented 2 weeks ago

Yeah, there is no harm in extending the heuristics to account for a wider range of possible shape names- especially if it is a fairly standardized/ubiquitous list that a large number of avatars make use of already.

sveken commented 1 week ago

Would this tie into https://github.com/Yellow-Dog-Man/Resonite-Issues/issues/1843 ?

shiftyscales commented 1 week ago

No, @sveken - That's a separate work item. This issue is just about adding blendshape name heuristic detection for avatars with that naming scheme to make it easier to set up the AvatarExpressionDriver, etc. with them. Until / unless #1843 is done, you'd still need to use an existing / supported face tracking solution, e.g. SRanipal with the Vive Facial Tracker / Vive Pro Eye for eye tracking, or the Quest Pro over Steam Link.