search

NVIDIA-ISAAC-ROS / isaac_ros_nvblox

NVIDIA-accelerated 3D scene reconstruction and Nav2 local costmap provider using nvblox
https://developer.nvidia.com/isaac-ros-gems
Apache License 2.0
409 stars 71 forks source link

Publishing full TSDF/ESDF maps #80

Closed simiken1234 closed 8 months ago

simiken1234 commented 8 months ago

Hello, are there any plans to publish the full esdf/tsdf maps on ROS topics as was the case with Voxblox? I really liked that implementation, also with the option to only publish updated blocks. With the currently published map slices it is hard to implement any type of 3D navigation on top of nvblox.

alexmillane commented 8 months ago

Hi there.

This is a really good question and something that I've thought a bit about. You're correct that at the moment there really is no good way for downstream nodes to consume the 3D map.

Our friends at NVIDIA research who use nvblox for path-planning for robot arms (and who use nvblox for 3D mapping), take the approach of building nvblox, the ROS independent library, into their path-planner. But ideally, we'd like people to be able to consume the map in ROS.

Unfortunately, I think that with the resolution and rates nvblox is typically run at, the approach we took in voxblox of transmitting map deltas is not ideal. In nvblox we'd have to copy 3D voxels back to CPU, potentially serialize them, and copy them back to the GPU on the receiver(s) end. I think that for typical usage this would be too inefficient.

The correct approach, I think, is to do messaging using shared GPU memory. This is possible in ROS2, see here for example for the isaac_ros flavor.

There is a chance we'll do this in the future, but we'll have to see where the project goes.

simiken1234 commented 8 months ago

I see. Seems like the shared GPU memory messaging would be a good approach then. Thanks for the advice, I will try to integrate nvblox directly into my project then.

For anyone who still wants a quick way to share the 3D map inefficiently, what I did was modify the pointcloud publishing function to publish a pointcloud of the full 3D map every time. On my end, I then stored the pointcloud in a hashmap which took a couple seconds for a pointcloud of about 0.5-1M points (working on CPU, no GPU accel) - usable for prototyping.

simiken1234 commented 8 months ago

I have to add that it would be nice to at least have a ros interface for querying individual voxel values. I know it wouldnt perform too well but it would be at least something