If Neos doesn't find English Visemes during avatar creation, fallback to Japanese & use DynamicVisemeDriver

[AshtonSparx]

Is your feature request related to a problem? Please describe.

With a ton of Japanese avatar's not having English visemes, a large portion of Pre-made avatar's don't work in Neos without a ton of know-how to manually assign Visemes & since they only use 5 vowels, the standard DirectVisemeDriver doesn't work without hacky methods being used.

Relevant issues

None I could find

Describe the solution you'd like

For the sake of Japanese users, provide auto support for Japanese viseme standards (This can include blinking with the Eye Manager). If there's neither that's found, it can skip trying to add visemes.

Describe alternatives you've considered

I'm unsure of any alternatives since DynamicVieseme already is capable of doing what I purpose

Additional context

Here's a chart I found that you can study! Deviantart Chart That includes all of the usual visemes such as a i u e o n and so on.

[shiftyscales]

Extending the heuristics would likely require a list of conventions/commonly used blendshape names in other languages.

Do you know how ubiquitous that list you provided is across common avatar bases?

[AshtonSparx]

I have tons of japanese avatars and they all use that Japanese naming standard that I provided above. For any other languages that don't use the English characters I am not aware of, however, if we have the system in place, i'm sure if other naming standards came fourth, people would be willing to include theirs. Here's my Windt Dragon's blendshapes from Booth as an example

[Frooxius]

Thanks for the chart, that'll help a lot, I'll add those names to the heuristics!

Is there a reason why are you proposing DynamicVisemeDriver though? I don't see why we can't just make the automatic mapping use that.

DynamicVisemeDriver is normally used when you want multiple visemes to affect single blendshape and use more complex combinations.

[AshtonSparx]

I purpose DynamicVisemeDriver because JP naming conventions usually only have roughly 5 blendshapes as featured above. DirectVisemeDriver requires more than that to fully utilize every syllable. So when you're talking, your mouth doesn't stop moving because it doesn't feature all of them. Here's an example of multiple syllables being tied to the same blendshapes

[Frooxius]

DirectVisemeDriver doesn't require all of them to be assigned to work. Is the mapping of different sounds to those 5 correct? Do you have a text version of this, I can't unfortunately copy those symbols from an image.

[AshtonSparx]

I know it doesn't require all of them, but the issue would be that, talking wouldn't always trigger mouth movement. You want your mouth to move as accurate as possible and continuously move during a conversation.

A: あ I: い U: う E: え O: お N: ん

I am unsure about blink however, if someone wants to share that, please do!

[AshtonSparx]

For Github Documentation, This has been added in Neos update 2021.7.11.1002 -- but to be left open for further implementations or bug fixes without the need for new Github issues. If the feature is satisfactory, closing the issue is an option. :)

[kazu0617]

I'll share with blink.

blink_both: まばたき

Also, if it is possible to include parameters related to emotions (I think it was provided in another component), the following information may be helpful.

smile: 笑顔 laugh: 笑い angry: おこ surprise: びっくり

[AshtonSparx]

You can manually assign that to the DirectVisemeDriver in the Laugh if that's what you mean

[kazu0617]

I checked from FrooxEngine, but it says:

    private const string JP_BLINK = "まばたき";
    private const string JP_WINK_RIGHT = "ウィンク右";
    private const string JP_WINK_LEFT = "ウィンク２";
    private const string JP_WINK_LEFT_ALT = "ウィンク";

In other says: ウィンク: Left Eye Blink ウィンク右: Right Eye Blink ウィンク２: Left Eye Blink (it override "ウィンク") ウィンク２右: Nothing <-????????

So if avatar sets 4 blendshapes, It will definitely make strange movements.

So plz add ウインク2右 to Right Eye Blink.

[shiftyscales]

I'd recommend making a separate/new issue for that specific request instead and closing this issue back up, @kazu0617. If you want, you could link back to this issue in your description of the problem.