Stereo Mode

[zhaozhongch]

Hi I am trying to use the stereo image in the demo bag for dense reconstruction but could not figure it out with the help in README.md

The command I am running is (run left image + depth is fine so kimera_semantic works properly in all)

roslaunch kimera_semantics_ros kimera_semantics.launch play_bag:=true metric_semantic_reconstruction:=false run_stereo_dense:=1

No error showing in the terminal. Just continuing updating the mesh without tsdf integration. The following is part of the terminal output.

[ INFO] [1600645927.696824786]: Loading nodelet /disparity_to_depth of type disparity_image_proc/depth_image to manager nodelet_manager with the following remappings:
[ INFO] [1600645927.697891757]: /depth_image -> /depth_image
[ INFO] [1600645927.697911344]: /disparity -> /disparity
[ INFO] [1600645927.697924606]: /left/camera_info -> /tesse/left_cam/camera_info
[ INFO] [1600645927.697938109]: /right/camera_info -> /tesse/right_cam/camera_info
[ INFO] [1600645927.698420176]: Initializing nodelet with 12 worker threads.
[ INFO] [1600645927.703761808]: Loading nodelet /cloudify of type depth_image_proc/point_cloud_xyzrgb to manager nodelet_manager with the following remappings:
[ INFO] [1600645927.704388888]: /depth_registered/image_rect -> /depth_image
[ INFO] [1600645927.704411498]: /depth_registered/points -> /semantic_pointcloud
[ INFO] [1600645927.704429927]: /rgb/camera_info -> /tesse/left_cam/camera_info
[ INFO] [1600645927.704444042]: /rgb/image_rect_color -> /tesse/left_cam/image_raw

Waiting 0.2 seconds after advertising topics...[ WARN] [1600645927.755778185]: Started in the global namespace! This is probably wrong. Start stereo_image_proc in the stereo namespace.
Example command-line usage:
    $ ROS_NAMESPACE=my_stereo rosrun stereo_image_proc stereo_image_proc
 done.

Hit space to toggle paused, or 's' to step.
[ INFO] [1600645928.062838229]: Updating mesh..........3 / 78.072170               
[ INFO] [1600645928.162767434]: Updating mesh..........3 / 78.072170               
[ INFO] [1600645928.263126438]: Updating mesh..........5 / 78.072170               
[ INFO] [1600645928.362990980]: Updating mesh..........2 / 78.072170               
[ INFO] [1600645928.462854218]: Updating mesh..........0 / 78.072170               
[ INFO] [1600645928.562461162]: Updating mesh..........7 / 78.072170               
[ WARN] [1600645928.597717500]: Color topic '/left/image_color' requested, but raw image data from topic '/tesse/left_cam/image_raw' is grayscale
[ INFO] [1600645928.663122907]: Updating mesh..........3 / 78.072170               
[ INFO] [1600645928.762962406]: Updating mesh..........0 / 78.072170               
[ INFO] [1600645928.862396811]: Updating mesh..........9 / 78.072170               
[ INFO] [1600645928.962913318]: Updating mesh..........8 / 78.072170               
[ INFO] [1600645929.062416152]: Updating mesh..........0 / 78.072170               
[ INFO] [1600645929.163045664]: Updating mesh..........1 / 78.072170               
[ INFO] [1600645929.263158255]: Updating mesh..........8 / 78.072170               
[ INFO] [1600645929.363039676]: Updating mesh..........1 / 78.072170               
[ INFO] [1600645929.462719128]: Updating mesh..........1 / 78.072170               
[ INFO] [1600645929.562801339]: Updating mesh..........9 / 78.072170               
[ INFO] [1600645929.662357726]: Updating mesh..........9 / 78.072170               
[ INFO] [1600645929.763141661]: Updating mesh..........2 / 78.072170               
[ INFO] [1600645929.862976349]: Updating mesh..........9 / 78.072170               
[ INFO] [1600645929.962583972]: Updating mesh..........0 / 78.072170     

I use the rviz to check and there is /points2 point cloud. But there is no depth image. Seems the disparity_image_proc is not working?

[zhaozhongch]

Still didn't figure out how to get the depth image using the disparity_image_proc. However, you don't need that package to let stereo mode work. Simply change the default setting is the launch file from

<remap from="pointcloud"                to="$(arg semantic_pointcloud)"/>

to

 <remap from="pointcloud"                to="/points2"/>

you can get the stereo mode work. The pointcloud is the input needed by the voxblox for reconstruction while the the points2 is the output from ros package stereo_image_proc

[sandilyasg]

@zhaozhongch Can you please explain what the launch file would look like for RGB, Depth, semantic segmentation frame coming directly from an RGBD camera and not getting depth from stereo images?