search

mmmmmm44 / VTuber-Python-Unity

An Implementation of VTuber (Both 3D and Live2D) using Python and Unity. Providing face movement tracking, eye blinking detection, iris detection and tracking and mouth movement tracking using CPU only.
MIT License
499 stars 84 forks source link

The reason why somttimes iris_image_points will run out of range #13

Open DannisMa opened 2 years ago

DannisMa commented 2 years ago

image

If you will list index out of range in main iris_image_points[j, 0] = faces[0][j + 468][0], you need to check your face detector init.

image

You need to set refine_landmarks be Ture. It mean detector will detect detail.

mmmmmm44 commented 2 years ago

Thank you for mentioning this. I would like to add on a bit.

After an update, there is one more API option available, which is the "refinelandmarks". From the official description, it "further refine the landmark coordinates around the eyes and lips" and output 10 more landmarks around iris for enhanced iris tracking, in the expense of a bit extra computational power. That's why there is one more for-loop to get the extra iris points without affecting the original 468 landmarks.

There are further details in their website about the extra landmarks. https://google.github.io/mediapipe/solutions/face_mesh#solution-apis