High quality scene reconstruction is possible?

[sentojh]

Hi. I'm trying to use reconstruction pipeline through python api. I'm using realsense l515 to get RGB-D sequences to feed into reconstruction system. I take pictures around my seat and want to get the reconstructed of many object seven if they have quite difficult geometry. In my case, the target object is mug cup.

Rarely, the mug cup is reconstructed well, but they are very few cases. Most cases are poor result in my opinion. Sometimes, cup is missing and the geometry is collapsed. I want to get correct and high quality reconstruction result.

I guess the reason comes from the make fragments or register fragments step due to alignment failure based on RGB-D odometry or loop closure checking to do global registration. But I could not find exact reason and solution to get real world scene reconstruction using rgbd camera.

Here is the result I uploaded both poor and good case Below result is gotten just taking the RGB-D sequences in front of cup(180 view only, but I think it is irrelevant. I tried all of thing I could)

quite good case

poor case(cup is missing)

poor case(cup handle is missing)

[sentojh]

I don't know why some detailed parts of object are collapsed. Is it possible or something should be wrong??

[theNded]

The pipeline is mainly designed for scene-scale reconstruction. Pose graph optimization performs an 'average' of poses that can over smooth the trajectory and the scene. So scene details can be blurry.

Potentially the SLAC module can refine details, but it is less stable and not guaranteed to converge well at current.

[sentojh]

Thanks for reply ! I would utilize the advice.

Actually, I have another question about it.

Could you briefly explain what is the difference in result between voxelhashing example and rgbd integration example in Open3D code?? I could not understand them in detail.....

[theNded]

VoxelHashing (now DenseSLAM) estimates poses and mapping simultaneously, while RGBD integration assumes known poses. The latter one is usually working with offline reconstruction. It is slower, but could be more accurate than DenseSLAM.