TianxingWu / OpenVHead

A 3D virtual head control system for VTuber in Unity with smooth motion and robust facial expressions
MIT License
439 stars 55 forks source link

Detection is glitchy and buggy #12

Open Schattenkobold opened 3 years ago

Schattenkobold commented 3 years ago

Hey,

I'm currently trying to make this work correctly and I'm having trouble with making it look anywhere nearly as good as what's shown in the demo video.

I tried using the programm on 2 different PCs and tested severall python, opencv and dlib versions including the ones which you list in the readme. I'm using Unity-Version 2018.4.33f1 for every try. Everytime the following happens when the python window opens and I see myself + the rotation detection marks:

In case of the CubeHanHan Model:

I found no info to problems like this in the readme or other Issues. Here is something I did which makes the glitches a bit less:

In ParameterServer.cs I found 2 blocks of code on the line 155 to 163. I commented the lines which look like this:

getheadRot.w = W_Kalman.kalman_filter(getheadRot.w, 8e-3f,5e-4f);

and uncommented the lines which look like this:

getheadRot.w = W_Kalman.kalman_filter(getheadRot.w, 1e-4f,1e-4f);

Did you experience anything like the problem I'm describing during development? I'm sadly too much of an beginner with facial detection, opencv and dlib to understand where things go wrong with the detection and/or the applying of the filters to figure it out myself.

Yours sincerely, S.