search

IntelRealSense / librealsense

Intel® RealSense™ SDK
https://www.intelrealsense.com/
Apache License 2.0
7.59k stars 4.82k forks source link

Kinect Fusion under opecv wrappers #3406

Closed vladsterz closed 5 years ago

vladsterz commented 5 years ago
Required Info
Camera Model D415
Operating System & Version Win 10
Platform PC/Raspberry Pi/ NVIDIA Jetson / etc..
SDK Version 2.17+

Issue Description

Has anyone built and run kinfu.exe ? I have much trouble with the latest realsense.dlls , in a way that I cannot run it. It crashes whenever a pipeline is created or is started.

dorodnic commented 5 years ago

Hi @vladsterz We will check. Its somewhat of a headache to build, requiring specialized build flags for both opencv and librealsense, so I admit we haven't really tested it lately. @AnnaRomanov please take a look Personally I found kinfu reconstruction somewhat disappointing (we also tried open3d) and I'm wondering if making small modifications to D435i+RTabMap (removing reliance on IMU and adjusting for D415) would produce better results.

vladsterz commented 5 years ago

Yes, I managed to build it after all, with librealsense 2.18.0. Results are indeed disappointing, even after I tried to run it with high accuracy preset. Fused data stop to make sense shortly after the recording starts and after a while it resets.

dorodnic commented 5 years ago

Were the results from open3D better?

Not sufficiently enough to publish. These methods seem to be extremely sensitive to rotations. I also tried OpenCV RGBD module. As a cheap pose estimator it performs pretty well, but does not help you to stitch resulting point-clouds.

vladsterz commented 5 years ago

Kinfu is not that bad under the configuration of small distance between foreground and background actually. image This is with high accuracy preset and the result is better than I imagined.

Jie2World commented 5 years ago

Hello @vladsterz Do you improve the recontruction? The poor result drive me crazy. If move the camera, the kinfu will reset soon which caused by getting zero matrix or nan/inf matrix A from getAB() function in icp->estimateTransformT()(fast-icp.cpp in opencv_rgbd). I try disable the optimization by adding setUseOptimized(false), which avoid getting a nan/inf matrix. But the zero matrix still exist. I am disappointed for these result. Have you try any library like PCL to reconstruct object with realsense d435? Best regards, Jie

vladsterz commented 5 years ago

Hello @Jie2World. I actually didn't looked in too much into the code or method itself. I made 2 modifications. i) I did change it to high accuracy in order have higher quality depth maps (laser power helps as well) and ii) I made it kind of offline process, reading depth maps from file.

Jie2World commented 5 years ago

@vladsterz Sorry, I can't understand the meaning of offline process. Do you mean that the program save the depth maps when kinfu fail and then read them after kinfu reset? Or you just read the depth maps from other place? I set it high accuracy but also get disappointing result. If move the camera, the point cloud will become blurred. Have you try other library such as PCL? PCL has same kinect fusion, but it works for kinect and I have no idea to use it with RealSense D435.

vladsterz commented 5 years ago

Hello @Jie2World. No , I haven't tried PCL implementation, actually I didn't know there is one.Offline process means that I read depth files from a directory and use them as an input to kinfu, it may be helpful if you want to pick the frames you like.

RealSenseCustomerSupport commented 5 years ago

@vladsterz Any other questions about this ticket? Looking forward to your update. Thanks!

RealSenseCustomerSupport commented 5 years ago

@vladsterz Any other questions?