search

nama-gatsuo / NuitrackForUE

Unreal Engine Plugin to wrap Nuitrack SDK ( skeleton tracking solution by 3DiVi )
13 stars 2 forks source link

Avatar screwing and non-rotation of the waist joint #2

Open nicolacapece opened 2 years ago

nicolacapece commented 2 years ago

Dear developer,

I try to use your UE4 plugin with Microsoft Azure Kinect and the Nuitrack AI license. However, there are several tracking problems, For example, the waist joint does not rotate, and consequently, the legs are screwed and the pose is unnatural. I attach two images to show the problem:

I activate the Nuitrack AI license in the Nuitrack IO under BP_Nuitrack. I don't understand if your plugin is configured only for the Nuitrack free license and consequently the tracking is less accurate.

It would also be useful to understand how to position the camera, at what height and distance from the user, and what is the optimal environmental lighting to increase the tracking accuracy.

Hope you can help me. Thank you in advance!

nama-gatsuo commented 2 years ago

Hi,

Thanks for pointing it out. Let me follow up one by one.

1) Crazily screwed or dumb joints Avatar control (Nuitrack Pose AnimGraphNode) is only using rotation data of joints (discarding position data). It means we use so-called Indirect Mapping for avatar control described in doc. If you see joints are wired, data that come from Nuitrack would be inconsistent. Data can be missing easily when a sensor lost tracking body parts. To prevent this behavior, limiting (clamping) rotation values to align natural angle for joints could be an option.

2) Compatible license I've tested with Nuitrack Pro which are working fine. Also I confirmed free license will work but timeout as expected within 3minutes and stop providing data. Though I haven't test Nuitrack AI license, it should respond as Pro do. If specific license cause wired behavior, that could be a bug.

3) Ideal condition for sensing skeleton You are right. In general, depth sensor captures infrared light reflecting from objects. Condition where infrared light level is too strong (such like outdoors on a sunny day, or surrounded too many sensors etc ) will cause bad result. Visibility for human eye is not so important. For a sensor figures should be in view without any chipping. Thus actors should stand 3~5m away without obstacles. Sensor height would be center of actor's height (1m or so)

Hope that helps you.