Can uLipSync be used for 3D characters?

[charlesckh1995]

I would like to use uLipSync for certain 3D models like Ready Player ME avatars and some other custom 3D models but these models seems not compatible with the plugin, is there any tutorial on this?

On the other hand, when there is background music playing, the character's mouth visemes are significantly interfered. Is there any way to eliminate the presence of a background noise/music when running in microhpone mode?

Thanks in advance for your sincere help.

[hecomi]

Sorry, I am not familiar with the specs of the Ready Player Me model, but if that model has BlendShapes, you can use LipSyncBlendShape to make it work.

If the model uses bones to move the mouth, you can write your own code to move it as described in Tips > Custom Event in the README.

using UnityEngine;
using uLipSync;

public class DebugPrintLipSyncInfo : MonoBehaviour
{
    public void OnLipSyncUpdate(LipSyncInfo info)
    {
        // move the mouth joint depending on the given info
    }
}

On the other hand, when there is background music playing, the character's mouth visemes are significantly interfered. Is there any way to eliminate the presence of a background noise/music when running in microhpone mode?

This is not possible as it is (and I don't think other lipsync plugins can do it as well). Please prepare voice-only sources. If this is difficult, please try an OSS that uses machine learning to separate the voices from the music or something.