search

OpenKinect / libfreenect2

Open source drivers for the Kinect for Windows v2 device
2.07k stars 748 forks source link

Depth Registration Question #569

Closed soulslicer closed 8 years ago

soulslicer commented 8 years ago

When using your API to get the Depth image. Is there any upscaling or preprocessing done on the image internally?

xlz commented 8 years ago

There is no upscaling by default. You can get upsamping one if you want. There is some median filtering and edge removal filtering.

soulslicer commented 8 years ago

Can I know the API to get this? Also, can I know which part of the code does these steps?

On Thu, Feb 11, 2016 at 12:42 PM, Lingzhu Xiang notifications@github.com wrote:

There is no upscaling by default. You can get upsamping one if you want. There is some median filtering and edge removal filtering.

— Reply to this email directly or view it on GitHub https://github.com/OpenKinect/libfreenect2/issues/569#issuecomment-182701388 .

xlz commented 8 years ago

Get what?

soulslicer commented 8 years ago

Essentially I'm trying to rewrite a point cloud generating software using OpenCV but on a Windows machine. I've taken the raw point cloud, and created a registered point cloud using projection matrices etc.

The Kinect sensor Windows API gives a 512x424 pixel image. However, the point cloud I generate is nowhere near is stable/dense as the one that is generated by the IAI_Kinect driver code which uses libfreenect2. The points also are alot more stable (do not move around alot) Their code looks very basic, and I'm trying to find out why the points generated from the depth image gotten from libfreenect2 seem superior.

It is clear that libfreenect2 is doing some form of upsampling. I'd like to replicate this on my Windows end. I've attempted to look through the code and have had trouble finding the section that does this. I would like to know exactly what upsampling techniques are used, whether OpenCV libraries were used to do these etc.?

Apart from that..I want to know if the libfreenect2 driver is superior to the Microsoft official drivers in terms of quality of the depth map etc.

xlz commented 8 years ago

iai_kinect2 does upsampling. Upsampling does not introduce more information.

Use bigdepth in https://openkinect.github.io/libfreenect2/classlibfreenect2_1_1Registration.html#a8be3a548ff440e30f5e89a4b053e04fa to get similar upsampled results.

soulslicer commented 8 years ago

Okay..but what algorithms are used in bigdepth step?

xlz commented 8 years ago

A 5x3 dilation filter.

soulslicer commented 8 years ago

Okay thanks. Is there any median/mean filtering done?

soulslicer commented 8 years ago

Lastly..what is your observation of depth maps generated from your system vs that of the Windows API?

xlz commented 8 years ago

Bilateral filtering is applied in depth processing https://openkinect.github.io/libfreenect2/structlibfreenect2_1_1Freenect2Device_1_1Config.html

I don't have any observation of the Microsoft SDK.

soulslicer commented 8 years ago

Thanks for answering my questions