search

microsoft / Azure-Kinect-Sensor-SDK

A cross platform (Linux and Windows) user mode SDK to read data from your Azure Kinect device.
https://Azure.com/Kinect
MIT License
1.49k stars 618 forks source link

Body index map is very inaccurate in many scenarios #949

Open cavisions opened 4 years ago

cavisions commented 4 years ago

Describe the bug

In many scenarios the body index map is very inaccurate. It has holes. Sometimes some body parts disappear completely. Sometimes the map is the sum of 2 users even when one user is standing about 1 meter behind the first one.

To Reproduce

This problem can be observed in the viewer application delivered with the body tracking SDK.

Expected behavior

I'd expect to have the body index map to be more accurate what will (in my opinion) influence also the quality of body joints placement. I'd expect it to work at least as good as it was working on Kinect v2.

Screenshots

1 1a 2 3 4 4a

Desktop (please complete the following information):

amantley commented 4 years ago

I am seeing something similar, here is a video that shows the shoulders disappearing when you are wearing a black t-shirt. if the same person puts on a blue coat then the appear correctly.

https://microsoft-my.sharepoint.com/:v:/p/anantley/Ect0R3LiIRpJsz_oA3Z2VcEBfPsZSi2LnUawof66b79Qsw?e=eLtEyz

cavisions commented 4 years ago

In my case it doesn't depend on what I'm wearing - the same effect is observed in many situations. It is also not dependent on the lighting conditions. It looks like it is so far poorly written. In the same conditions the Kinect v2 sensor is doing the job perfectly.

cavisions commented 4 years ago

It looks like the problem is not at all addressed in the Body Tracking SDK Version 1.0 what is very problematic. Do you have any plans to work on this issue?

eshirima commented 4 years ago

@cavisions hey, I was wondering if you could help me with this question since you seem to have it done.

fractalfantasy commented 4 years ago

@cavisions have a look at this feature request to implement kinect v2 ‘random forest’ model as an option - kinect v2 body tracking generated better body index, frame rate, latency and accuracy for dynamic movement:

https://feedback.azure.com/forums/920053-azure-kinect-dk/suggestions/39945454-legacy-body-tracking-like-kinect-v2

kinetronix commented 1 year ago

I've had some success in using a simple algorithm/heuristics for extracting an accurate the body index map from the current depth map and the last known body index map -- essentially start from body sdk-provided body index map and remove some unlikely pixels then spread the other pixels around the object unless close to the floor -- but it's not very robust so I have to limit how the user must position himself to interact. The advantage of k4a over Kinect V2 is the depth map is delivered faster though.

felixshing commented 1 month ago

I've had some success in using a simple algorithm/heuristics for extracting an accurate the body index map from the current depth map and the last known body index map -- essentially start from body sdk-provided body index map and remove some unlikely pixels then spread the other pixels around the object unless close to the floor -- but it's not very robust so I have to limit how the user must position himself to interact. The advantage of k4a over Kinect V2 is the depth map is delivered faster though.

Thank you for sharing! Would you mind sharing the code?