search

microsoft / Azure-Kinect-Sensor-SDK

A cross platform (Linux and Windows) user mode SDK to read data from your Azure Kinect device.
https://Azure.com/Kinect
MIT License
1.5k stars 620 forks source link

fastpointcloud example ply missing mesh face triangles (KinFu) #357

Closed dimitrijevic closed 5 years ago

dimitrijevic commented 5 years ago

The previous Kinect 2.0 SDK provided us with the ability to export a mesh beside just the point cloud, which seems to be currently missing in the K4A SDK, and as such will make us write more code which the SDK might solve.

If possible please provide the ability/example to generate a vertex colored mesh as well as point cloud.

Or, if this is not an SDK issue, could someone please provide some guidance on what alternative is best used, such as KinFu but its PCL implementation seems to only produce obj/mtl instead of a single ply from RGB images (or is there any other custom solutions for generating a TSDF cloud and a subsequent mesh)?

Thanks in advance, either way, really.

rabbitdaxi commented 5 years ago

Hi dimitrijevic, thank you for the feedback. Yes, we will look into this feature request. Beside of that, we can provide an example that leveraging the Azure Kinect Sensor SDK and OpenCV kinfu module (from contribute), it should give you live kinect fusion TSDF surface render as well as fused point cloud output.

dimitrijevic commented 5 years ago

Awesome, thank you, I was contemplating, using PCL myself maybe, when it hit 1.10 hopefully...but read somewhere their implementation's outdated, so OpenCV is probably an even better choice...I was hoping maybe to glimpse Microsoft's implementation though (I don't mean just the paper but to find out how their implementation seemed much better than PCL's)?

Once this and some sample data mkv repo for us without the sensor is available I'll try and maybe make a simple mex to call the C++ function that provides the vertex-colored ply mesh so the new Kinect can be used inside MATLAB as well just like with the Kin2 (although that implementation was missing also missing a colored mesh - which I'm hoping that OpenCV implementation has for ply as I haven't used it for a while I don't know if it's the same as PCL which only had obj/mtl for colored mesh so I'll look into it myself later this week to make sure maybe - but since Kin2 was still better to use without the Kinect v2 hooked up than MATLAB's Image Acquisition Toolbox which required the sensor to be present at first and which only provided the point cloud and not the KinFu mesh itself).

TIA all the same rabbitdaxi&cdedmonds

dimitrijevic commented 5 years ago

As promised (shameless self-github promo) (very simple MATLAB mex wrapper). Although if you guys are still contemplating a separate examples repo I'll gladly do a pull request there even though I know this is now still just quite simple code but even in this shape it may be of use to someone starting with MATLAB just as myself recently ... When I get around to upgrade my storage to build OpenCV and the SDK from sources I'll try and adapt the KinFu example but I'd like to ask first how much do you guys (and girls <- saw \\Build demo) plan to change the SDK, most importantly will the separate procedures for device and recording playback be unified and as such does it have a timeline, since that would make any MATLAB example deprecated IMHO, also will OpenCV be the recommended CV library or will you be rolling those processing features into the SDK in the future? TIA

rabbitdaxi commented 5 years ago

Just back from a conference, two follow up on this: 1) we published the sample that using Azure Kinect SDK with the OpenCV KinFu in our sample repo, which will be public soon 2) we have opened a feedback user voice to track the this feature request, https://feedback.azure.com/forums/920053-azure-kinect-dk/suggestions/37981348-kinect-fusion-support

dimitrijevic commented 5 years ago

No problem @rabbitdaxi, I kinda found/downloaded it, prior among pull requests already...thanks though....however, I haven't had the chance yet, to look at it and make mexFuncrion. As soon as I replace old SSD, and build the needed OpenCV contrib from sources, I'll be looking into it some more I promise.

As for the suggestions, just the one, for now...maybe, if possible, make a tool, for simulating the device, but that probably requires services, and the new SDK being multiplatform, I'm not sure if Kinect Studio's possible...that way no playback's needed, decreasing number of methods, should I put that on UserVoice?

rabbitdaxi commented 5 years ago

Hi @dimitrijevic , thank you for the suggestion, of course, any feedback is great, UserVoice is the place you can share feedback.

rabbitdaxi commented 5 years ago

Close the github issue based on the fact that we have a sample for opencv kinfu as one approach and we have the user voice to track a long term plan on Kinect Fusion support