search

devendrachaplot / Neural-SLAM

Pytorch code for ICLR-20 Paper "Learning to Explore using Active Neural SLAM"
http://www.cs.cmu.edu/~dchaplot/projects/neural-slam.html
MIT License
760 stars 144 forks source link

Regarding supervision for Mapper in Neural SLAM module and noise models #8

Closed kunalmessi10 closed 4 years ago

kunalmessi10 commented 4 years ago

Hi, I have the following questions regarding the code and your work.

  1. Can you give me some details about how to generate the ground truth for learning the mapper in the SLAM module?

  2. Firstly, as mentioned in the appendix, the vision range V is set as 64 i.e. 3.2m (each cell being 5cm in length). I understood logic of 64 being (3.2*100)/5, but was 64 or 3.2 picked as a design choice or there is some logic behind it.

  3. https://github.com/devendrachaplot/Neural-SLAM/blob/master/model.py#L196, here the grid_size = vr * 2, is this also a design choice or there is some reasoning behind it specifically being twice of vr.

  4. https://github.com/devendrachaplot/Neural-SLAM/blob/master/model.py#L206, lines 206-208, are 200 and 57. 29 for incorporating the noise models as mentioned in the paper or something else, if it's something else, could you briefly explain what exactly it is referring to, similar question for the get_new_pose_batch function as well https://github.com/devendrachaplot/Neural-SLAM/blob/master/model.py#L262 and this line https://github.com/devendrachaplot/Neural-SLAM/blob/master/model.py#L282.

  5. Lastly, to include the semantic categories as in your further ECCV 2020, and CVPR 2020 habitat challenge works, could you give me some details about how do you convert the depth map to point clouds, additionally if you could help me some details about the geometric computations used to convert the point cloud with the associated semantic labels to a 3-dimensional voxel map, it would be very helpful.

devendrachaplot commented 4 years ago

  1. The ground truth is generated in the environment class: https://github.com/devendrachaplot/Neural-SLAM/blob/master/env/habitat/exploration_env.py#L251 fp_proj and fp_explored is the ground truth in this line. The MapBuilder class is used to create and update ground truth maps: https://github.com/devendrachaplot/Neural-SLAM/blob/master/env/utils/map_builder.py

  2. This is a design choice.

  3. The grid size here has to be twice the vision range because the agent is assumed to be in the center of the grid.

  4. These constants are for converting meters to centimeters and radians to degrees.

  5. We are working on this code, we are planning to release the code sometime in the near future.