homuler / MediaPipeUnityPlugin

Unity plugin to run MediaPipe
MIT License
1.76k stars 460 forks source link

[Holistic] Extracting Coordinate to move a 3D avatar #902

Open enrikoyr opened 1 year ago

enrikoyr commented 1 year ago

Plugin Version or Commit ID

v.0.10.3

Unity Version

2021.3.11f1

Your Host OS

macOS Ventura M1

Target Platform

Android

Description

I want to use this plugin to animate a 3D avatar (.fbx format). The avatar already have all of its joints and landmark ready to go. I imported the avatar to the "Holistic" sample scene inside a new gameobject that I created called "Avatar". I also created a script inside this gameobject called "Animator.cs" that I will use to store the coordinate data from the holistic and use it to move my avatar.

Now the problem is, I can't find any reference to coordinate in the plugin. For example, in the "Holistic" scene, there's a gameObject hierarchy:

I opened some scripts that are attached to these gameObject (mainly "FaceLandmarkListAnnotation.cs") and it includes an array that contains numbered landmark of face (_connections). But try as I might, I can't extract any coordinate and print it on log from these list.

I also notice that there are lots of sphere gameObject that corresponds to each face landmark that is visible on the screen. I also notice that these sphere contains coordinate (standard unity coordinate on inspect window).

Is there any way to extract those coordinate? Can I just straight up attaching my 3D model to those sphere? Is my approach to this problem wrong? Please help me.

Code to Reproduce the issue

No response

Additional Context

No response

dgallegogarcia-capgemini commented 1 year ago

I'm wondering the same. I have the list of pose world landmarks, but these world gameobjects only have the position updated, and not the rotation, so when I move my hand, it works, but if I rotate it, it doesn't