Open jguan2984-1 opened 4 years ago
Hi @jguan2984-1! The global 3D Mesh is built using this package: https://github.com/MIT-SPARK/Kimera-Semantics
Hi @jguan2984-1! The global 3D Mesh is built using this package: https://github.com/MIT-SPARK/Kimera-Semantics Hi, thanks for the reply, I also tried to run the kimera-semantics, as far as I can see, the kimera-semantics could only load input from the simulated dataset you provided. It seems that the kimera-semantic and kimera-vio have not been incorporated together(which should be according to the paper)? Is there a way I could get the global 3D mesh by combining the kimera-vio and kimera semantic now? Thanks
They actually are! The launch file provided in Kimera-Semantics can be used for both simulation and kimera-VIO. Check the comments in the launch file itself:
<!-- Change sensor frame to:
- VIO's estimated base_link: `left_cam_base_link`
- Or, if you want to use simulator's ground-truth: `left_cam`
-->
Sorry for the confusion, we plan to document everything thoroughly but it takes a bit of time. Also, I will perhaps push a launch file in kimera-semantics to run everything in Euroc dataset to show how to connect with VIO.
They actually are! The launch file provided in Kimera-Semantics can be used for both simulation and kimera-VIO. Check the comments in the launch file itself:
<!-- Change sensor frame to: - VIO's estimated base_link: `left_cam_base_link` - Or, if you want to use simulator's ground-truth: `left_cam` -->
Sorry for the confusion, we plan to document everything thoroughly but it takes a bit of time. Also, I will perhaps push a launch file in kimera-semantics to run everything in Euroc dataset to show how to connect with VIO.
Thanks for the quick reply, a document and "pushing a launch file in kimera-semantics to run everything in Euroc dataset to show how to connect with VIO." would help a lot. Anyway, I will first try your suggestions.
They actually are! The launch file provided in Kimera-Semantics can be used for both simulation and kimera-VIO. Check the comments in the launch file itself:
<!-- Change sensor frame to: - VIO's estimated base_link: `left_cam_base_link` - Or, if you want to use simulator's ground-truth: `left_cam` -->
Sorry for the confusion, we plan to document everything thoroughly but it takes a bit of time. Also, I will perhaps push a launch file in kimera-semantics to run everything in Euroc dataset to show how to connect with VIO.
Hi, I changed the launch file for kimera semantic, but it is still not ok.
For kimera semantic, I don't need to load the bag file right?
It seem that kimera semantic could only get input(current position,cam_base link) from kimera vio,but could not get any single frame 3d mesh, it could only show the trajectory of the sensor, no global mesh, no single frame mesh.
I suppose I still need to change something in the launch file, but don't know how?
Hi @jguan2984-1, we don't use the per-frame 3D Mesh to generate the global 3D Mesh, instead, a dense stereo depth pipeline runs in parallel, providing accurate dense (pixel-wise) depth maps for Kimera-Semantics. Inside Kimera-Semantics launch file, there is a way to generate dense depth maps by calling a ROS package that implements StereoBM.
Hi, Thanks for the reply. So if I understand correctly, the kimera semantic only take the camera trajectory of kimera-vio as input, and the 3d mapping is built using stereoBM code inside kimera semantic itself.
What I am running now is
Oh, so, you need to explicitly tell kimera_semantics to run StereoBM: The default is set to false:
<arg name="run_stereo_dense" default="false"/>
See here.
And last thing, if you don't have semantic labels and just want to reconstruct the scene, set as well the metric_semantic_reconstruction
parameter to false.
<!-- If you just want to run 3D reconstruction without semantics, set this flag to false-->
<arg name="metric_semantic_reconstruction" default="true"/>
See here.
- Oh, so, you need to explicitly tell kimera_semantics to run StereoBM: The default is set to false:
<arg name="run_stereo_dense" default="false"/>
See here.
- And last thing, if you don't have semantic labels and just want to reconstruct the scene, set as well the
metric_semantic_reconstruction
parameter to false.<!-- If you just want to run 3D reconstruction without semantics, set this flag to false--> <arg name="metric_semantic_reconstruction" default="true"/>
See here. I still get the following error, it seem that I need to change the input topic for the kimera semantic? [ WARN] [1584064385.789873963]: The input topic '/tesse/left_cam/image_raw' is not yet advertised [ WARN] [1584064385.789985130]: The input topic '/tesse/left_cam/camera_info' is not yet advertised [ WARN] [1584064385.790024681]: The input topic '/tesse/right_cam/image_raw' is not yet advertised [ WARN] [1584064385.790067977]: The input topic '/tesse/right_cam/camera_info' is not yet advertised
Yes, so basically, summarizing, all the arg
elements in the launch file(s) need to be adapted to your particular use case and also all the topic names (aka the ones with remap
).
Are you using Euroc?
Yes, so basically, summarizing, all the
arg
elements in the launch file(s) need to be adapted to your particular use case and also all the topic names (aka the ones withremap
). Are you using Euroc? yes, I am using EUROC, could you provide a lauch file for euroc?
I'll put it in my todo list, I think another user also asked the same. Thank you!
ok, thanks..
ETA Saturday afternoon though, I'm a bit busy with other urgent things :+1:
Saturday afternoon will be great, it would be much better with also updated rviz file.
ETA Saturday afternoon though, I'm a bit busy with other urgent things
Hi, Any update about the launch file and rviz file?
I find there exists the launch file called kimera_semantics_euroc.launch
in the kimera-semantic package, and I try to follow your steps to run the program, but it still doesn't visualize the mesh.
@jguan2984-1 when run the kimera_semantics.launch
or kimera_semantics_euroc.launch
, the stereo_image_proc
ros node will start and accept the input camera_images and camera_infos, but the bag file doesn't include the camera infos. So it can't generate depth_image so I cannot see the mesh....
hi there, i am also facing to the same problem. I run rostopic echo /kimera_semantics_node/mesh
to see if there some output but none. And i also want to ask if the Kimera can do global mapping and export the map? thank you!
FYI, i am using D435i and trying to make it alive on the drone.
@ToniRV
I can see the single frame mesh in the riz, is the global 3D mesh implemented in this project? if so. how could I enable it and visulize it? Thanks