esperecyan / VRMConverterForVRChat

https://pokemori.booth.pm/items/1025226
Mozilla Public License 2.0
190 stars 23 forks source link

Viseme issue when ''convert for VRchat'' #66

Open SugarCrashhhh opened 8 months ago

SugarCrashhhh commented 8 months ago

I have an issue when converting for VRchat Fcl_MTH_U and O look fine, but then in the visemes vrc.v_ou and oh look exactly the same and they do not quite look like an ''oh'' nor ''ou''

Pictures of the issue: Unity_V9Tz2rTktv Unity_qVATJiD4W8

Pictures of Fcl_MTH_O and U: Unity_YNJ5yU3np7 Unity_vJ8HIdE1sX

I am using unity 2022 ver installed by the VCC, and running latest version of VRMConverterForVRChat

Is there a way to fix this? Thanks!

esperecyan commented 8 months ago

To generate viseme for VRChat, this extension uses the blend value of Blender add-on “Cats Blender Plugin” and only A, I, and O are used.

https://store.vket.com/items/122 https://github.com/absolute-quantum/cats-blender-plugin/blob/0.13.3/tools/viseme.py

SugarCrashhhh commented 8 months ago

I understand now that it works like Cats Blender Plugin, but this isn't right is it?

The vrc.v_ou and vrc.v_oh are the same, as you can see in the first two pictures So when you speak in VRchat, making an ''o'' or ''u'' sound looks awkward because of this

Could it be an issue with the newer Cats plugin or VRoid studio models? I used CATS plugin in blender before and the visemes looked right, now any model that goes through cats or VRMConverter has this issue

Do you get the same results when using the VRMConverter or Cats plugin?

Thank you

esperecyan commented 8 months ago

Yes, the algorithm has not changed for a long time and neither has the Cats Blender Plugin for a long time.

Although the algorithms for oh and ou are different, they may look the same depending on the Viseme from which they are converted. See below for the actual algorithm. https://github.com/absolute-quantum/cats-blender-plugin/blob/0.19.0/tools/viseme.py#L149-L159

There is no ability to customize this algorithm in this extension, but you can add a callback function that does the post-processing after the conversion is complete.