Exploration to PointGoal Transfer

[rpartsey]

Hi dear @devendrachaplot Thank you for open-sourcing your work! I enjoyed reading your paper "Learning To Explore Using Active Neural SLAM". The experiment setup and usage instructions are concise and easy to follow. I was able to run the code without any issues (except import errors fixed here https://github.com/devendrachaplot/Neural-SLAM/issues/1). The environment exploration visualizations generated by pre-trained models look amazing!

I'm struggling with Exploration to PointGoal Transfer. Basically, I understand the idea that the GlobalPolicy should be fixed to output PointGoal coordinates. But I fail to implement this idea in code.

• So the general question is: How to transform episode PointGoal coordinates to "local map" coordinates so that they could be used by the short-term goal planner? *I see that GlobalPolicy's output is a point in "local map" coordinates. My intuition is that PointGoal coordinates should be converted to "full map" coordinates and then to "local map" coordinates.

• Could you, please, also explain the math behind this expression? What does st mean? Why do we add 180 to orientation? https://github.com/devendrachaplot/Neural-SLAM/blob/e833e8f20c5a8d71552a5d22fb78ad6980cfc80c/env/habitat/exploration_env.py#L613

• And how get_grid function works? https://github.com/devendrachaplot/Neural-SLAM/blob/e833e8f20c5a8d71552a5d22fb78ad6980cfc80c/utils/model.py#L7 *I understand that it returns rotation and transition matrix. But, again, what is the math behind? I'm also confused with F.affine_grid function. And the similar transformation here https://github.com/devendrachaplot/Neural-SLAM/blob/e833e8f20c5a8d71552a5d22fb78ad6980cfc80c/model.py#L206

• And what is the intuition for applying get_new_pose_batch function? https://github.com/devendrachaplot/Neural-SLAM/blob/e833e8f20c5a8d71552a5d22fb78ad6980cfc80c/model.py#L262

[rpartsey]

I also wonder how you interpret alpha_beta_gamma angles returned by the as_euler_angles function (I mean angle - axis correspondence, which angle corresponds to which axis rotation?) and what is the intuition behind following if statement https://github.com/devendrachaplot/Neural-SLAM/blob/d5bc7886a4fdccae0287b7d0a26bf3800126b563/env/habitat/exploration_env.py#L375

[devendrachaplot]

Hi,

In order to transfer to the pointgoal task, you need to transform the relative pointgoal directions to map coordinates, pass these coordinates to the planner and add a check for stop action.

You will need to do the following:

• Add the following to reset function in env/habitat/exploration_env.py:

  dist, angle = obs["pointgoal"]
  x = int(dist*np.cos(angle)*20.0)
  y = int(dist*np.sin(angle)*20.0)
  self.pg_loc = [self.map_size_cm//2//args.map_resolution + y,
             self.map_size_cm//2//args.map_resolution + x]
  self.stop_next_action = 0

• Delete L471 in exploration_env.py (goal = inputs['goal']) and replace it by:

  goal = [self.pg_loc[0] - gx1, self.pg_loc[1] - gy1]
  if pu.get_l2_distance(start[0], goal[0], start[1], goal[1])*5 < 25:
          self.stop_next_action = 1

• Add a check to take stop action in the step function after L208 in exploration_env.py

  if self.stop_next_action == 1:
          action = 0

I might be missing a few things, let me know if the above does not work as expected.

Regarding the other questions, 'st' stands for spatial transformation. 'get_grid' and 'F.affine_grid' functions are used to get the parameters for the spatial transformation. You can check out the paper on spatial transformer networks (https://arxiv.org/pdf/1506.02025.pdf) and the pytorch tutorial (https://pytorch.org/tutorials/intermediate/spatial_transformer_tutorial.html) to get more details.

The 'get_new_pose_batch' function adds the relative pose change to the previous pose to the get the current pose. It computes current pose for a batch of inputs for efficiency.

The 'get_sim_location' function is converting the quaternion pose in the simulator to a single orientation of the agent in the top-down view.

Hope this helps.

[rpartsey]

Thanks a lot! The agent reaches the PointGoal as expected.

I've also added config_env.TASK.SENSORS.append("POINTGOAL_SENSOR") in Neural-SLAM/env/habitat/__init__.py after line 52. https://github.com/devendrachaplot/Neural-SLAM/blob/e833e8f20c5a8d71552a5d22fb78ad6980cfc80c/env/habitat/__init__.py#L52

The only thing that I'd like to clarify is how to properly handle the case when the agent reaches the goal in less than args.max_episode_length.

Currently, after the STOP action is called _episode_over property is set to True and if the step is less than args.max_episode_length, then the next envs.step(l_action) will be called on the finished episode and this results in AssertionError: Episode over, call reset before calling step. https://github.com/devendrachaplot/Neural-SLAM/blob/e833e8f20c5a8d71552a5d22fb78ad6980cfc80c/main.py#L329

[devendrachaplot]

The hacky/easy way of getting rid of this error is just waiting till the max_episode_length in the step function:

You can add the following at L198 in env/habitat/exploration_env.py:

if self._previous_action == 0:
    if self.timestep >= args.max_episode_length:
          done = True
    else:
          done = False
    null_state = np.zeros((3, self.args.frame_height, self.args.frame_width))
    return null_state, 0., done, self.info

The better way, which will require a lot of code change, is to pass is to set done=True when self._previous_action == 0 in step function and then checking which thread has episode finished in main.py (at L371) and resetting map for a thread when it is finished. You will also need to sync this with local steps (because map get shifted between global steps). This should not change the results, it will only help in reducing the runtime.

Some more notes:

• If you want to run the Pointgoal task with GPS+Compass (without pose noise), you need to run main.py with --noisy_actions 0, --noisy_odometry 0 and --use_pose_estimation 0 arguments
• The pose estimator is trained with our pose noise models based on real-world data, it is not expected to work with the pose noise models implemented in Habitat currently.
• To get the best results during evaluation, run with --map_size_cm 3000 and --global_downscaling 1 arguments. This will increase the runtime slightly.

[rpartsey]

@devendrachaplot Thank you!