search

microsoft / Azure-Kinect-Sensor-SDK

A cross platform (Linux and Windows) user mode SDK to read data from your Azure Kinect device.
https://Azure.com/Kinect
MIT License
1.47k stars 614 forks source link

Vaguely human shapes generate body tracking skeleton #1106

Open billpottle opened 4 years ago

billpottle commented 4 years ago

Describe the bug When recording in our martial arts school, the body tracking sdk generates a skeleton for this Body Opponent Bag (BOB) bob

To Reproduce You would need the pictured object to reproduce, but I am happy to share files.

Expected behavior I expect the SDK would use the fact that there is no lower body, no arms, no movement, and no difference from ambient room temperature to conclude that the object is not a living person.

Desktop (please complete the following information):

Additional context We can of course ignore the skeleton in post processing, but I believe this represents an area of improvement for the SDK.

Certainly, the SDK should take into account that people could be missing various parts of their bodies. However, the combination of all the factors, especially the lack of body heat, should be enough to generate a correct result.

SeanStephensenAIRM commented 4 years ago

Azure Kinect doesn't have a temperature sensor, so it will have trouble detecting a presence or lack of distinct body temperature.

billpottle commented 4 years ago

@SeanStephensenAIRM - I see. I thought the IR information coming from the depth camera could be used to get some temperature data.

SeanStephensenAIRM commented 4 years ago

@billpottle The Kinect sensor operates in the NIR region, specifically at 850nm. Thermal infrared is transmitted at 8000-15000 nm, far above the range of the Kinect.

qm13 commented 4 years ago

The DNN has no way of knowing if the detected person is real or not. It also cannot determine if the person (or training dummy in this case) has no arms of that the arms are obscured. We are investigating options to reduce false positives. We are currently collecting recordings of scenes with objects that could be confused as humans but no real humans in view. Could you capture a scene with the training dummy? If so reply with your email address and I will send you instructions on sharing the recording with Microsoft.

To work around the problem you could use the joint confidence numbers to potentially eliminate the skeleton.

billpottle commented 4 years ago

@qm13 - Thank you. It's not a huge issue for us, but just wanted to do what we can to improve the tech. I already have signed the release from sharing with a previous issue, so I sent a link to the recording directly to your email.

qm13 commented 4 years ago

@billpottle Thanks for the capture.