Neleac / MesekaiUnity

webcam motion tracking avatar for Vtubing and multiplayer games
https://neleac.itch.io/mesekai
GNU General Public License v3.0
31 stars 12 forks source link

Feature: legs and hips tracking #1

Open FarzanAlipour opened 1 year ago

FarzanAlipour commented 1 year ago

Hey, I checked the repo and it is super nice! I wanted to know if you have any plans to track legs and hips in this package P.N : I have used the same logic for legs and hips but it is not as good as arms.

Neleac commented 1 year ago

I also tried using the same logic and the results are not great, see fullbody branch. I suspect the problem is the initial landmark-avatar alignment. I simply rotate the landmarks to match the hips by comparing 2 vectors, but this should actually be a 3D coordinate alignment calculation.

FarzanAlipour commented 1 year ago

sorry for the delay. I have checked the fullbody branch, and yes the results are not good. I achieved a better result with caching the spine and hip upward direction in the Start and then calculate the FromToRotation for the cached vector and mediapipe upward for hips and spine ( which are calculated in a separate method)