search

microsoft / Azure-Kinect-Sensor-SDK

A cross platform (Linux and Windows) user mode SDK to read data from your Azure Kinect device.
https://Azure.com/Kinect
MIT License
1.49k stars 619 forks source link

How to get point cloud data of human bodies only #1346

Closed Koring12 closed 3 years ago

Koring12 commented 4 years ago

Hi,

The project I am working on requires point cloud data of only human body excluding background and other objects. I am wondering if it is possible to get a set of point cloud data of human body using Azure Kinect Body Tracking SDK?

Any help would be welcomed!

Thanks!

DWD3 commented 4 years ago

Hi, in result object generated by the Body tracking SDK, there is an image called 'body index map', that image will highlight all the points on the depth image that belong to a human body.

You can find more info here https://docs.microsoft.com/en-us/azure/kinect-dk/body-index-map

qm13 commented 4 years ago

@DWD3 is correct in that you want to use instance segmentation to isolate the human(s) from the scene. You could use the Azure Kinect Body Tracking SDK to achieve this but it may be overkill. Instead you could use a publicly available semantic or instance segmentation model to process the IR image to isolate the pixels associated with human(s) and use this info in rendering the point cloud.

DWD3 commented 4 years ago

@DWD3 is correct in that you want to use instance segmentation to isolate the human(s) from the scene. You could use the Azure Kinect Body Tracking SDK to achieve this but it may be overkill. Instead you could use a publicly available semantic or instance segmentation model to process the IR image to isolate the pixels associated with human(s) and use this info in rendering the point cloud.

Hi @qm13 Thanks for the response. Do you have any suggestion on which instance segmentation model to use? Currently I am use the body tracking API to do the job, but as you said, it's a bit of an overkill.

qm13 commented 4 years ago

Sorry but I cannot make a specific recommendation.

Koring12 commented 3 years ago

Thank you guys for your help I used Azure Kinect Body Tracking SDK to solve this problem!

socketing commented 2 years ago

@DWD3是正确的，因为您希望使用实例分割将人员与场景隔离开来。你可以使用 Azure Kinect Body Tracking SDK 来实现此目的，但这可能有些过头了。相反，您可以使用公开可用的语义或实例分割模型来处理 IR 图像，以隔离与人关联的像素，并使用此信息来渲染点云。

嗨@qm13 感谢您的回复。您对使用哪种实例分段模型有什么建议吗？目前，我正在使用身体跟踪API来完成这项工作，但正如你所说，这有点矫枉过正。

@DWD3 Hi,Listen to your tone, you seem to be Chinese? If so, we can talk about 3D reconstruction. I hope you can tell me your contact information.