How to integrate Nanomap with rotors_simualtor?

[JThanat]

Hi!, I have read through you paper on Nanomap and it is really interesting approach. I have managed to pull down docker image provided in the document. However, I am a bit curious on how to integrate this Nanomap with the rotors_simulator. I found rotor_simulator. in your build folder in the docker, but I could not figure out how Nanomap can actually be integrated into rotor_simulator

I am now working on uav collision avoidance project using stereo camera and I think that your work here is really amazing. So I would like to see how could it be integrated into simulator so that I could try feeding point cloud generated with stereo camera instead of depth camera.

Thank you in advance :smile:

[peteflorence]

Hey @JThanat . Glad you're interested. rotor_simulator must be in the docker image due to something from Voxblox or Ewok. The purpose of the docker image was to benchmark against bagged data, rather than provide a live simulation environment. We haven't used rotor_simulator.

It shouldn't be too bad to integrate NanoMap with any simulation environment of your choice?

Our own simulation environment is described a little bit more in a previous paper: http://groups.csail.mit.edu/robotics-center/public_papers/Florence16.pdf We use Unity to build worlds and do rgb/depth sim, and Drake to do dynamics simulation. It's a little hard for us to have this full sim environment be open source because it depends on many components that are hard to distribute, like the Unity worlds. @psiorx @jakeware

But yes any sim environment should work OK as long as NanoMap can get point clouds and poses. I'd recommend checking out AirSim from Microsoft.

Good luck and let us know how it goes!

[JThanat]

Thank you so much @peteflorence for your clarification.

I have read through your code and your paper on Nanomap. If I understand correctly, to integrate Nanomap into uav path planning and obstacle avoidance in unknown environment simulation, I need to use Nanomap with other motion planners (both global and local planner) as mentioned in issue #3 right?

Your master thesis is also very interesting I may have to research more on the thesis and your repo on the topic.

As of now, I may need to figure out a bit more on Nanomap integration. If it is done in anyway, I might be able to share it in the future as well.

[peteflorence]

Hey @JThanat, yes NanoMap is meant to have a clear in/out specification that you can ask it for uncertainty-aware approximate nearest neighbor queries over a recent history of depth camera information. That way, it can be used with any type of trajectory-library or sampling-based motion planner.

You don't need to have separate local and global motion planners but in practice that is a good idea to do so. NanoMap is a good candidate for the map representation for local planning.

Also for reference the full master's thesis pdf is at this link, and has expanded sections on background and related work that might be helpful.

Let us know if you have any other questions?