dense_slam_gui.py and Realsense l515 Error: Singular 6x6 linear system detected, tracking failed.

[olagt]

Checklist

• [X] I have searched for similar issues.
• [X] For Python issues, I have tested with the latest development wheel.
• [X] I have checked the release documentation and the latest documentation (for master branch).

Describe the issue

When I run example Open3D/examples/python/t_reconstruction_system/./dense_slam_gui.py with test data ie. indoor_apartment then it works properly. But when I collect data using real-sense l515 with: python3 realsense_recorder.py --record_imgs --output_folder ./roomA01022022/ with stream definition : config.enable_stream(rs.stream.depth, 1024, 768, rs.format.z16, 30) config.enable_stream(rs.stream.color, 1280, 720, rs.format.bgr8, 30) then program crashes at first frames with error

RuntimeError: [Open3D Error] (void open3d::t::pipelines::kernel::DecodeAndSolve6x6(const open3d::core::Tensor&, open3d::core::Tensor&, float&, int&)) /home/ola/Proj2/Open3D/cpp/open3d/t/pipelines/kernel/TransformationConverter.cpp:145: Singular 6x6 linear system detected, tracking failed.

(Open3D14_1) ola@ola-ASUS-AI:~/Proj2/Open3D/examples/python/t_reconstruction_system$ python ./dense_slam_gui.py --config /media/ola/Int2TB/AI_DATA3/RS_IK/roomA_trunc/roomA_trunc.yml 
FEngine (64 bits) created at 0x7f777a3a9010 (threading is enabled)
FEngine resolved backend: OpenGL
Exception in thread UpdateMain:
Traceback (most recent call last):
  File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.8/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "./dense_slam_gui.py", line 375, in update_main
    result = self.model.track_frame_to_model(
RuntimeError: [Open3D Error] (void open3d::t::pipelines::kernel::DecodeAndSolve6x6(const open3d::core::Tensor&, open3d::core::Tensor&, float&, int&)) /home/ola/Proj2/Open3D/cpp/open3d/t/pipelines/kernel/TransformationConverter.cpp:145: Singular 6x6 linear system detected, tracking failed.

I tried it with multiple recording with different resolution - but the error stays . I also record almost static scenario when sensor is no moving - same error. Depth Scale is set according to real sense sensor - 4000 ( depth_scale = depth_sensor.get_depth_scale() )

It crashes either with CPU:0 / CUDA:0

Steps to reproduce the bug

1) collect data using realsense_recorder with l515 connected
2) modyfie yml exmaple config replacing depth_scale from 1000 to 4000 
3) run python ./dense_slam_gui.py --config media/ola/Int2TB/AI_DATA3/RS_IK/roomA_trunc/roomA_trunc.yml 
4) programs crashes 

name: Default reconstruction system config
fragment_size: 100
device: CUDA:0
engine: tensor
multiprocessing: false
path_dataset: '/media/ola/Int2TB/AI_DATA3/RS_IK/roomA_trunc'
depth_folder: depth
color_folder: color
path_intrinsic: '/media/ola/Int2TB/AI_DATA3/RS_IK/roomA_trunc/camera_intrinsic.json'
depth_min: 0.1
depth_max: 3.0
depth_scale: 4000.0
odometry_method: hybrid
odometry_loop_interval: 10
odometry_loop_weight: 0.1
odometry_distance_thr: 0.07
icp_method: colored
icp_voxelsize: 0.05
icp_distance_thr: 0.07
global_registration_method: ransac
registration_loop_weight: 0.1
integration_mode: color
voxel_size: 0.0058
sdf_trunc: 0.04
block_count: 40000
surface_weight_thr: 3.0

Error message

(Open3D14_1) ola@ola-ASUS-AI:~/Proj2/Open3D/examples/python/t_reconstruction_system$ python ./dense_slam_gui.py --config /media/ola/Int2TB/AI_DATA3/RS_IK/roomA_trunc/roomA_trunc.yml FEngine (64 bits) created at 0x7f577db02010 (threading is enabled) FEngine resolved backend: OpenGL Exception in thread UpdateMain: Traceback (most recent call last): File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner self.run() File "/usr/lib/python3.8/threading.py", line 870, in run self._target(*self._args, **self._kwargs) File "./dense_slam_gui.py", line 375, in update_main result = self.model.track_frame_to_model( RuntimeError: [Open3D Error] (void open3d::t::pipelines::kernel::DecodeAndSolve6x6(const open3d::core::Tensor&, open3d::core::Tensor&, float&, int&)) /home/ola/Proj2/Open3D/cpp/open3d/t/pipelines/kernel/TransformationConverter.cpp:145: Singular 6x6 linear system detected, tracking failed.

Saving model to output.npz... Finished. Extracting and saving mesh to output.ply... [Open3D WARNING] Write PLY failed: mesh has 0 vertices. Finished. Saving trajectory to output.log... Finished.

Expected behavior

No response

Open3D, Python and System information

- Operating system: ubunt 20.04
- Python version: Python 3.8.10
- Open3D version: 0.14.1+f6ab689b5
- System type: x84 
- Is this remote workstation?: no
- How did you install Open3D?: build from source
- Compiler version (if built from source): gcc 9.3

Additional information

Data from l515 as tested against ./dense_slam_gui.py roomA_trunc.zip

[theNded]

I have identified the issue. It was caused by the missing scale factor in the pipeline: https://github.com/isl-org/Open3D/blob/6aa17206f273df77c54f11b6efc97aef125d5517/cpp/open3d/t/pipelines/slam/Model.cpp#L111 It will be formally fixed in #4670. Temporarily you may change this line to

voxel_grid_.Integrate(frustum_block_coords_, depth, color, intrinsic, extrinsic, depth_scale, depth_max);

and recompile the system by make install-pip-package. After the fix here is the output:

[olagt]

Thank you very much ! Now it works :)

[nasireddypavan]

Hi @theNded ,

Facing the same tracking failure with the latest version. I'm using Real-sense D435I to record.

Error:

RuntimeError: [Open3D Error] (void open3d::t::pipelines::kernel::DecodeAndSolve6x6(const open3d::core::Tensor&, open3d::core::Tensor&, float&, int&)) /root/Open3D/cpp/open3d/t/pipelines/kernel/TransformationConverter.cpp:146: Singular 6x6 linear system detected, tracking failed.

using default config file.