search

emilianavt / VSeeFaceReleases

This is a release repository used for posting releases of VSeeFace
118 stars 6 forks source link

No Lipsync or mouth movement in VSeeFace on Windows 10 #1

Closed vivi90 closed 2 years ago

vivi90 commented 3 years ago

Again an issue with VSeeFace, instead of OpenSeeFace, sorry :-)

On Linux with Wine the Lipsync and mouth movement while speaking works great for me. But now i have switched to Windows 10 and there is no Mouth movement working. The expressions by training working but no mouth movement by speaking. I selected my connected USB microphone. In Audacity it works and i can hear me after playback. VSeeFace detects also my microphone but it doesn't work there. Tested also to increase the "Lip sync audio overdrive gain" but no change.

vivi90 commented 3 years ago

Okay, seems it depends on the "Mouth size reduction". I need to use this setting at at least 50% because otherwise the lower jaw leaves the model. But now it's hardly noticeable that i am speaking.

emilianavt commented 3 years ago

A high mouth size reduction will definitely cause issues like this. I recommend disabling mouth tracking in the expression settings for the affected expression instead. For a real fix, adjust the expressions in VRoid (0.12+) or Unity.

For general VSeeFace support, please try our discord, contact me on Twitter through DMs or even use the VSeeFaceReleases repository. It doesn't really belong here.

Edit: I moved the issues from the OpenSeeFace repository.

vivi90 commented 3 years ago

For a real fix, adjust the expressions in VRoid (0.12+)

I am using VRoid Studio 0.13.1. Which expression setting do i need to change?

or even use the VSeeFaceReleases repository.

Oh great, didn't know that, sorry :-)

emilianavt commented 3 years ago

I believe the teeth clipping out of the jaw mainly happens with the Joy and Surprised expressions, so you could reduce the size of the mouth for those in VRoid Studio.

vivi90 commented 3 years ago

I believe the teeth clipping out of the jaw mainly happens with the Joy and Surprised expressions

Just disabling mouth sync for these expressions helped a lot.

so you could reduce the size of the mouth for those in VRoid Studio.

Yes, this was the solution, thanks! :-)

By the way, is there an easy way to enable the teeth fangs inside VSeeFace? Would be great.

emilianavt commented 3 years ago

Fangs need to be set up on the model. For some reason VRoid doesn't export them properly, so some adjustments on the blendshape settings need to be made in Unity (e.g. turning up the fang blendshapes for all VRM blend shape clips that open the mouth).

vivi90 commented 3 years ago

@emilianavt Okay, lip sync doesn't work on Windows 10 for me. It just detects the mouth movement by face detection but nothing by microphone.

On Linux with Wine it works now. But not on Windows 10. Tried to use the factory reset but doesn't work :-(

What sort of diagnostic information do you need?

emilianavt commented 3 years ago

The lipsync troublshooting part of the website lists most things to check. Most important would be that the input is on the left channel of the selected audio device and whether there are any FMOD errors.

vivi90 commented 3 years ago

The lipsync troublshooting part of the website lists most things to check. Most important would be that the input is on the left channel of the selected audio device and whether there are any FMOD errors.

Okay, i will check that. Until now i can just say, that it seems to be an stereo microphone, because in OBS there are 2 channels displayed at the VU meter. I am using the Microphone Array of the Microsoft Kinect v1