search

MIT-SPARK / Kimera-VIO-ROS

ROS wrapper for Kimera-VIO
BSD 2-Clause "Simplified" License
369 stars 154 forks source link

"Missing landmark information to build 3D Mesh" using Realsense D435i #126

Open mtmercado opened 3 years ago

mtmercado commented 3 years ago

Hello! Thank you for your work on this library! It is very interesting! I started playing around with it and in need of help. I'm trying use Kimera VIO and Semantics with Intel Realsense D435i. However, I'm encountering this warning whenever I run the kimera_vio launch file:

W0203 14:13:26.420764 4725 Mesher.cpp:1380] Missing landmark information to build 3D Mesh.

I tried to run kimera_semantics as well however, I'm not getting the desired result. I'm assuming the 3D mesh is a data required in Kimera Semantics, is that right? If so, how can I specify the landmark information for the 3D mesh to build successfully?

Thank you!

Logs: kimera vio

ailab-cv@ailabcv-NUC8i7BEH:~$ roslaunch kimera_vio_ros kimera_vio_ros_realsense_IR.launch run_stereo_dense:=true
... logging to /home/ailab-cv/.ros/log/6d4b41b6-65de-11eb-a604-04ea56db17cb/roslaunch-ailabcv-NUC8i7BEH-4689.log
Checking log directory for disk usage. This may take a while.
Press Ctrl-C to interrupt
Done checking log file disk usage. Usage is <1GB.

started roslaunch server http://ailabcv-NUC8i7BEH:39771/

SUMMARY
========

PARAMETERS
 * /kimera_vio_ros/kimera_vio_ros_node/base_link_frame_id: base_link
 * /kimera_vio_ros/kimera_vio_ros_node/left_cam_frame_id: left_cam
 * /kimera_vio_ros/kimera_vio_ros_node/map_frame_id: map
 * /kimera_vio_ros/kimera_vio_ros_node/mono_ransac_threshold: 30
 * /kimera_vio_ros/kimera_vio_ros_node/online_run: True
 * /kimera_vio_ros/kimera_vio_ros_node/params_folder_path: /home/ailab-cv/ca...
 * /kimera_vio_ros/kimera_vio_ros_node/position_det_threshold: 0.3
 * /kimera_vio_ros/kimera_vio_ros_node/right_cam_frame_id: right_cam
 * /kimera_vio_ros/kimera_vio_ros_node/stereo_ransac_threshold: 20
 * /kimera_vio_ros/kimera_vio_ros_node/use_lcd: False
 * /kimera_vio_ros/kimera_vio_ros_node/use_online_cam_params: False
 * /kimera_vio_ros/kimera_vio_ros_node/velocity_det_threshold: 0.1
 * /kimera_vio_ros/kimera_vio_ros_node/world_frame_id: world
 * /kimera_vio_ros/posegraph_viewer/frame_id: world
 * /rosdistro: melodic
 * /rosversion: 1.14.10
 * /use_sim_time: False

NODES
  /
    velo_link_broadcaster (tf/static_transform_publisher)
  /kimera_vio_ros/
    kimera_vio_ros_node (kimera_vio_ros/kimera_vio_ros_node)
    posegraph_viewer (pose_graph_tools/visualizer_node)

ROS_MASTER_URI=http://localhost:11311

process[kimera_vio_ros/kimera_vio_ros_node-1]: started with pid [4704]
process[kimera_vio_ros/posegraph_viewer-2]: started with pid [4705]
process[velo_link_broadcaster-3]: started with pid [4706]
[ INFO] [1612329205.947763145]: Initializing pose graph visualizer
I0203 14:13:26.096387  4704 RegularVioBackEnd.cpp:114] Using Regular VIO backend.
I0203 14:13:26.096485  4704 PipelineModule.h:449] MISO Pipeline Module: Display has no output queue registered.
I0203 14:13:26.096644  4704 Pipeline.cpp:479] Pipeline Modules launched (parallel_run set to 1).
I0203 14:13:26.096861  4729 Pipeline.h:199] Spinning Kimera-VIO.
I0203 14:13:26.096849  4704 KimeraVioRos.cpp:117] Statistics
-----------                                  #  Log Hz  {avg     +- std    }    [min,max]
Display [ms]                                    0   
VioBackEnd [ms]                                 0   
VioFrontEnd [ms]                                0   
Visualizer [ms]                                 0   
backend_input_queue Size [#]                    0   
data_provider_left_frame_queue Size [#]         0   
data_provider_right_frame_queue Size [#]        0   
display_input_queue Size [#]                    0   
mesher_backend Size [#]                         0   
mesher_frontend Size [#]                        0   
stereo_frontend_input_queue Size [#]            0   
visualizer_backend_queue Size [#]               0   
visualizer_frontend_queue Size [#]              0   
visualizer_mesher_queue Size [#]                0   
I0203 14:13:26.419855  4724 VioBackEnd.h:168] ------------------- Initialize Pipeline: timestamp = 3224655983534148216--------------------
W0203 14:13:26.419875  4724 InitializationFromImu.cpp:25] InitializationFromImu: assumes that the vehicle is stationary and upright along some axis,and gravity vector is along a single axis!
W0203 14:13:26.420764  4725 Mesher.cpp:1380] Missing landmark information to build 3D Mesh.
I0203 14:13:26.481225  4724 RegularVioBackEnd.cpp:212] Tracker has a LOW_DISPARITY status.
W0203 14:13:26.481602  4725 Mesher.cpp:1380] Missing landmark information to build 3D Mesh.
I0203 14:13:26.561883  4724 RegularVioBackEnd.cpp:212] Tracker has a LOW_DISPARITY status.
W0203 14:13:26.562310  4725 Mesher.cpp:1380] Missing landmark information to build 3D Mesh.
I0203 14:13:26.657797  4724 RegularVioBackEnd.cpp:212] Tracker has a LOW_DISPARITY status.
W0203 14:13:26.658249  4725 Mesher.cpp:1380] Missing landmark information to build 3D Mesh.
W0203 14:13:26.773025  4723 FeatureDetector.cpp:150] No corners extracted for frame with id: 13
I0203 14:13:26.773860  4724 RegularVioBackEnd.cpp:212] Tracker has a LOW_DISPARITY status.
W0203 14:13:26.774258  4725 Mesher.cpp:1380] Missing landmark information to build 3D Mesh.
I0203 14:13:26.857635  4724 RegularVioBackEnd.cpp:212] Tracker has a LOW_DISPARITY status.
W0203 14:13:26.858050  4725 Mesher.cpp:1380] Missing landmark information to build 3D Mesh.
I0203 14:13:26.959893  4724 RegularVioBackEnd.cpp:212] Tracker has a LOW_DISPARITY status.
W0203 14:13:26.960284  4725 Mesher.cpp:1380] Missing landmark information to build 3D Mesh.
I0203 14:13:27.072741  4724 RegularVioBackEnd.cpp:212] Tracker has a LOW_DISPARITY status.
W0203 14:13:27.073794  4725 Mesher.cpp:1380] Missing landmark information to build 3D Mesh.
I0203 14:13:27.096922  4704 KimeraVioRos.cpp:117] Statistics
-----------                                  #  Log Hz  {avg     +- std    }    [min,max]
Data Provider [ms]                              0   
Display [ms]                                   15   15.3381 {1.46667 +- 5.68038}    [0,22]
Mesher [ms]                                     8   8.18165 {0.125000 +- 0.353553}  [0,1]
VioBackEnd [ms]                                 8   8.19239 {0.00000 +- 0.00000}    [0,0]
VioFrontEnd Frame Rate [ms]                    15   22.6023 {2.80000 +- 0.774597}   [2,4]
VioFrontEnd Keyframe Rate [ms]                  7   10.7232 {17.5714 +- 1.81265}    [15,20]
VioFrontEnd [ms]                               23   23.3126 {8.73913 +- 9.16127}    [2,36]
Visualizer [ms]                                 8   8.17989 {0.00000 +- 0.00000}    [0,0]
backend_input_queue Size [#]                    8   7.98631 {1.00000 +- 0.00000}    [1,1]
data_provider_left_frame_queue Size [#]        25   25.3270 {1.00000 +- 0.00000}    [1,1]
data_provider_right_frame_queue Size [#]       25   25.3263 {1.00000 +- 0.00000}    [1,1]
display_input_queue Size [#]                   15   14.9431 {1.00000 +- 0.00000}    [1,1]
mesher_backend Size [#]                         8   8.19097 {1.00000 +- 0.00000}    [1,1]
mesher_frontend Size [#]                       23   23.3020 {1.91304 +- 0.848155}   [1,3]
stereo_frontend_input_queue Size [#]           23   22.7877 {1.00000 +- 0.00000}    [1,1]
visualizer_backend_queue Size [#]               8   8.19100 {1.00000 +- 0.00000}    [1,1]
visualizer_frontend_queue Size [#]             23   23.3022 {1.91304 +- 0.848155}   [1,3]
visualizer_mesher_queue Size [#]                8   8.17826 {1.00000 +- 0.00000}    [1,1]
I0203 14:13:27.165833  4724 RegularVioBackEnd.cpp:212] Tracker has a LOW_DISPARITY status.
W0203 14:13:27.166347  4725 Mesher.cpp:1380] Missing landmark information to build 3D Mesh.
I0203 14:13:27.260519  4724 RegularVioBackEnd.cpp:212] Tracker has a LOW_DISPARITY status.
W0203 14:13:27.260944  4725 Mesher.cpp:1380] Missing landmark information to build 3D Mesh.
I0203 14:13:27.357152  4724 RegularVioBackEnd.cpp:212] Tracker has a LOW_DISPARITY status.
W0203 14:13:27.357560  4725 Mesher.cpp:1380] Missing landmark information to build 3D Mesh.
I0203 14:13:27.459976  4724 RegularVioBackEnd.cpp:212] Tracker has a LOW_DISPARITY status.
W0203 14:13:27.460459  4725 Mesher.cpp:1380] Missing landmark information to build 3D Mesh.
I0203 14:13:27.557564  4724 RegularVioBackEnd.cpp:212] Tracker has a LOW_DISPARITY status.
W0203 14:13:27.558034  4725 Mesher.cpp:1380] Missing landmark information to build 3D Mesh.
I0203 14:13:27.684725  4724 RegularVioBackEnd.cpp:212] Tracker has a LOW_DISPARITY status.
W0203 14:13:27.685144  4725 Mesher.cpp:1380] Missing landmark information to build 3D Mesh.
I0203 14:13:27.759567  4724 RegularVioBackEnd.cpp:212] Tracker has a LOW_DISPARITY status.
W0203 14:13:27.760056  4725 Mesher.cpp:1380] Missing landmark information to build 3D Mesh.
^CI0203 14:13:27.860966  4724 RegularVioBackEnd.cpp:212] Tracker has a LOW_DISPARITY status.
W0203 14:13:27.861502  4725 Mesher.cpp:1380] Missing landmark information to build 3D Mesh.

kimera semantics

... logging to /home/ailab-cv/.ros/log/72152ac0-65e0-11eb-a604-04ea56db17cb/roslaunch-ailabcv-NUC8i7BEH-5661.log
Checking log directory for disk usage. This may take a while.
Press Ctrl-C to interrupt
Done checking log file disk usage. Usage is <1GB.

started roslaunch server http://ailabcv-NUC8i7BEH:41043/

SUMMARY
========

CLEAR PARAMETERS
 * /kimera_semantics_node/

PARAMETERS
 * /cloudify/queue_size: 20
 * /kimera_semantics_node/color_mode: lambert_color
 * /kimera_semantics_node/dynamic_semantic_labels: [20]
 * /kimera_semantics_node/enable_icp: False
 * /kimera_semantics_node/icp_iterations: 10
 * /kimera_semantics_node/max_ray_length_m: 5
 * /kimera_semantics_node/mesh_filename: /home/ailab-cv/ca...
 * /kimera_semantics_node/method: fast
 * /kimera_semantics_node/min_time_between_msgs_sec: 0.2
 * /kimera_semantics_node/publish_pointclouds: False
 * /kimera_semantics_node/semantic_color_mode: semantic
 * /kimera_semantics_node/semantic_label_2_color_csv_filepath: /home/ailab-cv/ca...
 * /kimera_semantics_node/semantic_measurement_probability: 0.8
 * /kimera_semantics_node/sensor_frame: left_cam
 * /kimera_semantics_node/slice_level: 1.0
 * /kimera_semantics_node/tsdf_voxel_size: 0.02
 * /kimera_semantics_node/tsdf_voxels_per_side: 16
 * /kimera_semantics_node/update_mesh_every_n_sec: 0.1
 * /kimera_semantics_node/use_const_weight: False
 * /kimera_semantics_node/use_freespace_pointcloud: False
 * /kimera_semantics_node/use_tf_transforms: True
 * /kimera_semantics_node/verbose: True
 * /kimera_semantics_node/voxel_carving_enabled: True
 * /rosdistro: melodic
 * /rosversion: 1.14.10
 * /use_sim_time: False

NODES
  /
    cloudify (nodelet/nodelet)
    kimera_semantics_node (voxblox_ros/tsdf_server)
    nodelet_manager (nodelet/nodelet)

ROS_MASTER_URI=http://localhost:11311

process[nodelet_manager-1]: started with pid [5677]
process[cloudify-2]: started with pid [5678]
process[kimera_semantics_node-3]: started with pid [5679]
[ INFO] [1612330048.302067468]: Loading nodelet /cloudify of type depth_image_proc/point_cloud_xyzrgb to manager nodelet_manager with the following remappings:
[ INFO] [1612330048.302637143]: /depth_registered/image_rect -> /kimera/depth_registered/image_rect_raw
[ INFO] [1612330048.302654935]: /depth_registered/points -> /semantic_pointcloud
[ INFO] [1612330048.302670181]: /rgb/camera_info -> /camera/color/camera_info
[ INFO] [1612330048.302684132]: /rgb/image_rect_color -> /camera/color/image_raw
[ INFO] [1612330048.303633830]: waitForService: Service [/nodelet_manager/load_nodelet] has not been advertised, waiting...
[ INFO] [1612330048.322386107]: Initializing nodelet with 8 worker threads.
[ INFO] [1612330048.324319301]: waitForService: Service [/nodelet_manager/load_nodelet] is now available.
[ INFO] [1612330048.497208818]: Updating mesh.
[ INFO] [1612330048.597140305]: Updating mesh.
[ INFO] [1612330048.697146803]: Updating mesh.
[ INFO] [1612330048.797159197]: Updating mesh.
Ravi3191 commented 3 years ago

I am having the same problem here. Did you resolve the issue?

BBlumhofer commented 2 years ago

I am having the same issue. Did anyone found a solution for this ?