search

MIT-SPARK / Kimera-VIO-ROS

ROS wrapper for Kimera-VIO
BSD 2-Clause "Simplified" License
376 stars 157 forks source link

Is global 3D mesh (covering the entire trajectory) enabled? #46

Open jguan2984-1 opened 4 years ago

jguan2984-1 commented 4 years ago

I can see the single frame mesh in the riz, is the global 3D mesh implemented in this project? if so. how could I enable it and visulize it? Thanks

ToniRV commented 4 years ago

Hi @jguan2984-1! The global 3D Mesh is built using this package: https://github.com/MIT-SPARK/Kimera-Semantics

jguan2984-1 commented 4 years ago

Hi @jguan2984-1! The global 3D Mesh is built using this package: https://github.com/MIT-SPARK/Kimera-Semantics Hi, thanks for the reply, I also tried to run the kimera-semantics, as far as I can see, the kimera-semantics could only load input from the simulated dataset you provided. It seems that the kimera-semantic and kimera-vio have not been incorporated together(which should be according to the paper)? Is there a way I could get the global 3D mesh by combining the kimera-vio and kimera semantic now? Thanks

ToniRV commented 4 years ago

They actually are! The launch file provided in Kimera-Semantics can be used for both simulation and kimera-VIO. Check the comments in the launch file itself:

  <!-- Change sensor frame to:
   - VIO's estimated base_link: `left_cam_base_link`
   - Or, if you want to use simulator's ground-truth: `left_cam`
  -->

Sorry for the confusion, we plan to document everything thoroughly but it takes a bit of time. Also, I will perhaps push a launch file in kimera-semantics to run everything in Euroc dataset to show how to connect with VIO.

jguan2984-1 commented 4 years ago

They actually are! The launch file provided in Kimera-Semantics can be used for both simulation and kimera-VIO. Check the comments in the launch file itself:

  <!-- Change sensor frame to:
   - VIO's estimated base_link: `left_cam_base_link`
   - Or, if you want to use simulator's ground-truth: `left_cam`
  -->

Sorry for the confusion, we plan to document everything thoroughly but it takes a bit of time. Also, I will perhaps push a launch file in kimera-semantics to run everything in Euroc dataset to show how to connect with VIO.

Thanks for the quick reply, a document and "pushing a launch file in kimera-semantics to run everything in Euroc dataset to show how to connect with VIO." would help a lot. Anyway, I will first try your suggestions.

jguan2984-1 commented 4 years ago

They actually are! The launch file provided in Kimera-Semantics can be used for both simulation and kimera-VIO. Check the comments in the launch file itself:

  <!-- Change sensor frame to:
   - VIO's estimated base_link: `left_cam_base_link`
   - Or, if you want to use simulator's ground-truth: `left_cam`
  -->

Sorry for the confusion, we plan to document everything thoroughly but it takes a bit of time. Also, I will perhaps push a launch file in kimera-semantics to run everything in Euroc dataset to show how to connect with VIO.

Hi, I changed the launch file for kimera semantic, but it is still not ok. For kimera semantic, I don't need to load the bag file right?
It seem that kimera semantic could only get input(current position,cam_base link) from kimera vio,but could not get any single frame 3d mesh, it could only show the trajectory of the sensor, no global mesh, no single frame mesh.

I suppose I still need to change something in the launch file, but don't know how?

ToniRV commented 4 years ago

Hi @jguan2984-1, we don't use the per-frame 3D Mesh to generate the global 3D Mesh, instead, a dense stereo depth pipeline runs in parallel, providing accurate dense (pixel-wise) depth maps for Kimera-Semantics. Inside Kimera-Semantics launch file, there is a way to generate dense depth maps by calling a ROS package that implements StereoBM.

jguan2984-1 commented 4 years ago

Screenshot from 2020-03-04 09-39-17 Hi， Thanks for the reply. So if I understand correctly, the kimera semantic only take the camera trajectory of kimera-vio as input, and the 3d mapping is built using stereoBM code inside kimera semantic itself.

What I am running now is

  1. inside kimera vio workspace run :roscore
  2. inside kimera vio workspace run :roslaunch kimera_vio_ros kimera_ros_euroc.launch
  3. inside kimera semantic workspace run :roslaunch kimera_semantics_ros kimera_semantics.launch
  4. inside kimera semantic workspace run : rviz -d $(rospack find kimera_semantics_ros)/rviz/kimera_semantics_gt.rviz
  5. inside kimera vio workspace run :rosbag play --clock /media/nvidia/3466-6434/jguan/euroc/MH_02_easy.bag The result is shown in the figure, it seems that the kimera semantic can receive the trajectory input "left_cam_base_link" from kimera vio, but can not get input the stereo input. Except
ToniRV commented 4 years ago

  1. Oh, so, you need to explicitly tell kimera_semantics to run StereoBM: The default is set to false:

    <arg name="run_stereo_dense"     default="false"/>

    See here.

  2. And last thing, if you don't have semantic labels and just want to reconstruct the scene, set as well the metric_semantic_reconstruction parameter to false.

    <!-- If you just want to run 3D reconstruction without semantics, set this flag to false-->
<arg name="metric_semantic_reconstruction" default="true"/>

    See here.

jguan2984-1 commented 4 years ago
  1. Oh, so, you need to explicitly tell kimera_semantics to run StereoBM: The default is set to false:
<arg name="run_stereo_dense"     default="false"/>

See here.

  1. And last thing, if you don't have semantic labels and just want to reconstruct the scene, set as well the metric_semantic_reconstruction parameter to false.
  <!-- If you just want to run 3D reconstruction without semantics, set this flag to false-->
  <arg name="metric_semantic_reconstruction" default="true"/>

See here. I still get the following error, it seem that I need to change the input topic for the kimera semantic? [ WARN] [1584064385.789873963]: The input topic '/tesse/left_cam/image_raw' is not yet advertised [ WARN] [1584064385.789985130]: The input topic '/tesse/left_cam/camera_info' is not yet advertised [ WARN] [1584064385.790024681]: The input topic '/tesse/right_cam/image_raw' is not yet advertised [ WARN] [1584064385.790067977]: The input topic '/tesse/right_cam/camera_info' is not yet advertised

ToniRV commented 4 years ago

Yes, so basically, summarizing, all the arg elements in the launch file(s) need to be adapted to your particular use case and also all the topic names (aka the ones with remap). Are you using Euroc?

jguan2984-1 commented 4 years ago

Yes, so basically, summarizing, all the arg elements in the launch file(s) need to be adapted to your particular use case and also all the topic names (aka the ones with remap). Are you using Euroc? yes, I am using EUROC, could you provide a lauch file for euroc?

ToniRV commented 4 years ago

I'll put it in my todo list, I think another user also asked the same. Thank you!

jguan2984-1 commented 4 years ago

ok, thanks..

ToniRV commented 4 years ago

ETA Saturday afternoon though, I'm a bit busy with other urgent things :+1:

jguan2984-1 commented 4 years ago

Saturday afternoon will be great, it would be much better with also updated rviz file.

jguan2984-1 commented 4 years ago

ETA Saturday afternoon though, I'm a bit busy with other urgent things

Hi, Any update about the launch file and rviz file?

tftangming commented 4 years ago

I find there exists the launch file called kimera_semantics_euroc.launch in the kimera-semantic package, and I try to follow your steps to run the program, but it still doesn't visualize the mesh.

tftangming commented 4 years ago

@jguan2984-1 when run the kimera_semantics.launch or kimera_semantics_euroc.launch, the stereo_image_proc ros node will start and accept the input camera_images and camera_infos, but the bag file doesn't include the camera infos. So it can't generate depth_image so I cannot see the mesh....

DragonOnGit commented 1 year ago

hi there, i am also facing to the same problem. I run rostopic echo /kimera_semantics_node/mesh to see if there some output but none. And i also want to ask if the Kimera can do global mapping and export the map? thank you! FYI, i am using D435i and trying to make it alive on the drone.

@ToniRV