search

MIT-SPARK / Kimera-Semantics

Real-Time 3D Semantic Reconstruction from 2D data
BSD 2-Clause "Simplified" License
640 stars 136 forks source link

How to combine Kimera-Semantics with Kimera-VIO-ROS ? #35

Closed YvonneZhao0305 closed 3 years ago

YvonneZhao0305 commented 4 years ago

Hi, thank you for your excellent work! @ToniRV I followed the instructions from your video tutorial "Metric-Semantic SLAM with Kimera: A Hands-On Tutorial"

roslaunch kimera_vio_ros kimera_vio_ros_tesse.launch run_stereo_dense:=1 rosbag play kimera_semantics_demo.bag --pause --clock --rate 2 roslaunch kimera_semantics_ros kimera_semantics.launch rviz -d $(rospack find kimera_vio_ros)/rviz/kimera_vio_tesse.rviz

and I stuck in the first command

started core service [/rosout] ERROR: cannot launch node of type [stereo_image_proc/stereo_image_proc]: stereo_image_proc ROS path [0]=/opt/ros/kinetic/share/ros ROS path [1]=/home/zss/Workspace/kimera_vio_ws/src/catkin_simple ROS path [2]=/home/zss/Workspace/kimera_vio_ws/src/cmake_external_project_catkin ROS path [3]=/home/zss/Workspace/kimera_vio_ws/src/dbow2_catkin ROS path [4]=/home/zss/Workspace/kimera_vio_ws/src/disparity_image_proc ROS path [5]=/home/zss/Workspace/kimera_vio_ws/src/gflags_catkin ROS path [6]=/home/zss/Workspace/kimera_vio_ws/src/glog_catkin ROS path [7]=/home/zss/Workspace/kimera_vio_ws/src/gtsam ROS path [8]=/home/zss/Workspace/kimera_vio_ws/src/image_undistort ROS path [9]=/home/zss/Workspace/kimera_vio_ws/src/Kimera-RPGO ROS path [10]=/home/zss/Workspace/kimera_vio_ws/src/kimera_rviz_markers ROS path [11]=/home/zss/Workspace/kimera_vio_ws/src/Kimera-VIO ROS path [12]=/home/zss/Workspace/kimera_vio_ws/src/mesh_rviz_plugins ROS path [13]=/home/zss/Workspace/kimera_vio_ws/src/opencv3_catkin ROS path [14]=/home/zss/Workspace/kimera_vio_ws/src/opengv_catkin ROS path [15]=/home/zss/Workspace/kimera_vio_ws/src/pose_graph_tools ROS path [16]=/home/zss/Workspace/kimera_vio_ws/src/Kimera-VIO-ROS ROS path [17]=/home/zss/Workspace/kimera_semantic_ws/src/catkin_boost_python_buildtool/catkin_boost_python_buildtool ROS path [18]=/home/zss/Workspace/kimera_semantic_ws/src/catkin_simple ROS path [19]=/home/zss/Workspace/kimera_semantic_ws/src/catkin_boost_python_buildtool/catkin_boost_python_buildtool_test ROS path [20]=/home/zss/Workspace/kimera_semantic_ws/src/eigen_catkin ROS path [21]=/home/zss/Workspace/kimera_semantic_ws/src/gflags_catkin ROS path [22]=/home/zss/Workspace/kimera_semantic_ws/src/glog_catkin ROS path [23]=/home/zss/Workspace/kimera_semantic_ws/src/eigen_checks ROS path [24]=/home/zss/Workspace/kimera_semantic_ws/src/Kimera-Semantics/kimera ROS path [25]=/home/zss/Workspace/kimera_semantic_ws/src/minkindr/minkindr ROS path [26]=/home/zss/Workspace/kimera_semantic_ws/src/minkindr_ros/minkindr_conversions ROS path [27]=/home/zss/Workspace/kimera_semantic_ws/src/pointcloud_recolor ROS path [28]=/home/zss/Workspace/kimera_semantic_ws/src/protobuf_catkin ROS path [29]=/home/zss/Workspace/kimera_semantic_ws/src/schweizer_messer/python_module ROS path [30]=/home/zss/Workspace/kimera_semantic_ws/src/schweizer_messer/numpy_eigen ROS path [31]=/home/zss/Workspace/kimera_semantic_ws/src/minkindr/minkindr_python ROS path [32]=/home/zss/Workspace/kimera_semantic_ws/src/schweizer_messer/sm_common ROS path [33]=/home/zss/Workspace/kimera_semantic_ws/src/schweizer_messer/sm_boost ROS path [34]=/home/zss/Workspace/kimera_semantic_ws/src/schweizer_messer/sm_deprecation ROS path [35]=/home/zss/Workspace/kimera_semantic_ws/src/schweizer_messer/sm_doc ROS path [36]=/home/zss/Workspace/kimera_semantic_ws/src/schweizer_messer/sm_logging ROS path [37]=/home/zss/Workspace/kimera_semantic_ws/src/schweizer_messer/sm_matrix_archive ROS path [38]=/home/zss/Workspace/kimera_semantic_ws/src/schweizer_messer/sm_opencv ROS path [39]=/home/zss/Workspace/kimera_semantic_ws/src/schweizer_messer/sm_property_tree ROS path [40]=/home/zss/Workspace/kimera_semantic_ws/src/schweizer_messer/sm_random ROS path [41]=/home/zss/Workspace/kimera_semantic_ws/src/schweizer_messer/sm_eigen ROS path [42]=/home/zss/Workspace/kimera_semantic_ws/src/schweizer_messer/sm_kinematics ROS path [43]=/home/zss/Workspace/kimera_semantic_ws/src/schweizer_messer/sm_matlab ROS path [44]=/home/zss/Workspace/kimera_semantic_ws/src/schweizer_messer/sm_timing ROS path [45]=/home/zss/Workspace/kimera_semantic_ws/src/schweizer_messer/sm_value_store ROS path [46]=/home/zss/Workspace/kimera_semantic_ws/src/schweizer_messer/sm_python ROS path [47]=/home/zss/Workspace/kimera_semantic_ws/src/voxblox/voxblox ROS path [48]=/home/zss/Workspace/kimera_semantic_ws/src/Kimera-Semantics/kimera_semantics ROS path [49]=/home/zss/Workspace/kimera_semantic_ws/src/voxblox/voxblox_msgs ROS path [50]=/home/zss/Workspace/kimera_semantic_ws/src/voxblox/voxblox_rviz_plugin ROS path [51]=/home/zss/Workspace/kimera_semantic_ws/src/voxblox/voxblox_ros ROS path [52]=/home/zss/Workspace/kimera_semantic_ws/src/Kimera-Semantics/kimera_semantics_ros ROS path [53]=/opt/ros/kinetic/share process[kimera_vio_ros/kimera_vio_ros_node-3]: started with pid [2197] process[kimera_vio_ros/posegraph_viewer-4]: started with pid [2198] process[gt_logger_node-5]: started with pid [2199] [ INFO] [1596597287.980138782]: Initializing pose graph visualizer I0805 11:14:48.164175 2197 Logger.cpp:58] Opening output file: output_frontend_stats.csv I0805 11:14:48.164247 2197 Logger.cpp:58] Opening output file: output_frontend_ransac_mono.csv I0805 11:14:48.164260 2197 Logger.cpp:58] Opening output file: output_frontend_ransac_stereo.csv I0805 11:14:48.164572 2197 Logger.cpp:58] Opening output file: traj_vio.csv I0805 11:14:48.164587 2197 Logger.cpp:58] Opening output file: output_smartFactors.csv I0805 11:14:48.164597 2197 Logger.cpp:58] Opening output file: output_pim_navstates.csv I0805 11:14:48.164605 2197 Logger.cpp:58] Opening output file: output_backendFactors.csv I0805 11:14:48.164614 2197 Logger.cpp:58] Opening output file: output_backendTiming.csv I0805 11:14:48.164865 2197 RegularVioBackEnd.cpp:114] Using Regular VIO backend. I0805 11:14:48.164985 2197 PipelineModule.h:449] MISO Pipeline Module: Display has no output queue registered. F0805 11:14:48.165057 2197 LoopClosureDetector.cpp:93] Check failed: f_vocab.good() LoopClosureDetector: Incorrect vocabulary path: /home/zss/Workspace/kimera_vio_ws/src/Kimera-VIO/vocabulary/ORBvoc.yml Check failure stack trace: @ 0x7f85670fe43d google::LogMessage::Fail() @ 0x7f8567100253 google::LogMessage::SendToLog() @ 0x7f85670fdfcb google::LogMessage::Flush() @ 0x7f8567100c3e google::LogMessageFatal::~LogMessageFatal() @ 0x7f855c0fcc21 VIO::LoopClosureDetector::LoopClosureDetector() @ 0x7f855c0dffee VIO::Pipeline::Pipeline() @ 0x7f85666a475b _ZN3VIO11make_uniqueINS_8PipelineEJRNS_9VioParamsESt10unique_ptrINS_13RosVisualizerESt14default_deleteIS5_EES4_INS_10RosDisplayES6_IS9_EEEEES4_IT_S6_ISCEEDpOT0 @ 0x7f856669c1e6 VIO::KimeraVioRos::runKimeraVio() @ 0x40bfbc main @ 0x7f8563805830 libc_start_main @ 0x40c6b9 _start [kimera_vio_ros/kimera_vio_ros_node-3] process has died [pid 2197, exit code -6, cmd /home/zss/Workspace/kimera_vio_ws/devel/lib/kimera_vio_ros/kimera_vio_ros_node --use_lcd=true --vocabulary_path=/home/zss/Workspace/kimera_vio_ws/src/Kimera-VIO/vocabulary/ORBvoc.yml --flagfile=/home/zss/Workspace/kimera_vio_ws/src/Kimera-VIO/params/Tesse/flags/Mesher.flags --flagfile=/home/zss/Workspace/kimera_vio_ws/src/Kimera-VIO/params/Tesse/flags/VioBackEnd.flags --flagfile=/home/zss/Workspace/kimera_vio_ws/src/Kimera-VIO/params/Tesse/flags/RegularVioBackEnd.flags --flagfile=/home/zss/Workspace/kimera_vio_ws/src/Kimera-VIO/params/Tesse/flags/Visualizer3D.flags --logtostderr=1 --colorlogtostderr=1 --log_prefix=1 --v=0 --log_output=true --log_euroc_gt_data=true --output_path=/home/zss/Workspace/kimera_vio_ws/src/Kimera-VIO-ROS/output_logs/Tesse --viz_type=0 --visualize=true left_cam/image_raw:=/tesse/left_cam/image_raw right_cam/image_raw:=/tesse/right_cam/image_raw imu:=/tesse/imu gt_odom:=/tesse/odom left_cam/camera_info:=/tesse/left_cam/camera_info right_cam/camera_info:=/tesse/right_cam/camera_info reinit_flag:=reinit_flag reinit_pose:=reinit_pose odometry:=odometry resiliency:=resiliency imu_bias:=imu_bias optimized_trajectory:=optimized_trajectory pose_graph:=pose_graph mesh:=mesh frontend_stats:=frontend_stats debug_mesh_img/image_raw:=debug_mesh_img/image_raw feature_tracks/image_raw:=feature_tracks/image_raw time_horizon_pointcloud:=time_horizon_pointcloud name:=kimera_vio_ros_node __log:=/home/zss/.ros/log/d0f98c6c-d6c9-11ea-8975-4889e7b44eca/kimera_vio_ros-kimera_vio_ros_node-3.log]. log file: /home/zss/.ros/log/d0f98c6c-d6c9-11ea-8975-4889e7b44eca/kimera_vio_ros-kimera_vio_ros_node-3*.log

so I changed commands as following

roslaunch kimera_vio_ros kimera_vio_ros_euroc.launch rosbag play ./V1_01_easy.bag --pause --clock --rate 2 roslaunch kimera_semantics_ros kimera_semantics_euroc.launch rviz -d $(rospack find kimera_vio_ros)/rviz/kimera_vio_euroc.rviz

Screenshot from 2020-08-05 11-24-09

It works well at this time, but generate local(not global) 3D mesh without semantic information. Could you give me some advice?

ToniRV commented 3 years ago

Done! See new readme here: #44