Connect Facebook's AI Habitat simulator with ROS so that motion planning researchers can take advantage of photorealistic 3D environments and easily simulate tasks such as visual SLAM and navigation.
Navigating Habitat environment with ROS navigation package: https://youtu.be/VYfZ4wghpRA
Navigating Gazebo (ROS' simulator) environment with Habitat trained agent: https://youtu.be/EaU_a6MIeIE
Navigating Habitat environment with Habitat trained agent with ROS in the loop: https://youtu.be/eYywGkWd_0E
Slide deck for demo purposes: https://1drv.ms/p/s!AhI7FLXI6kP7hfA3lsetrnm2y0Uq9g
ROS Kinetic (http://wiki.ros.org/kinetic)
*depthimage_to_laserscan
*laser_scan_matcher
*hector_slam [needed during mapping and navigation]
Anaconda (https://www.anaconda.com/distribution/)
cd ~/catkin_ws && catkin_make
(replace catkin_ws with your own workspace name if applicable)sudo apt-get install ros-kinetic-depthimage-to-laserscan
, sudo apt-get install ros-kinetic-laser-scan-matcher
, and sudo apt-get install ros-kinetic-hector-slam
. If you encounter errors relating to "cannot find package", this link helped me solve the problem https://answers.ros.org/question/325039/apt-update-fails-cannot-install-pkgs-key-not-working/The following steps ensure you can run Anaconda along side ROS (we need Anaconda because Habitat requires python>=3.6 while ROS requires python2)
pip install rospkg
so the file habitat-api/habitat_ros/hab_ros_plant.py can be ran from your Python3.6 Anaconda environmentThe following picture shows how I modified my ~/.bashrc file to complete steps 7 and 8
Activate your anaconda python>=3.6 and cd into habitat-api's root directory
Source your ROS related setup.bash files
cd into the habitat-api directory
Run python habitat_ros/hab_ros_interface.py
to run the node that publishes on habitat sensor reading topics and subscribes to the /cmd_vel topic
Deactivate Anaconda as you won't need python>=3.6 anymore to interact with the Habitat back-end (This step is hacky. I usually just go inside my ~/.bashrc and comment/uncomment the lines related to Anaconda manually depending on whether or not I need Anaconda)
close and re-open your terminal
Source your ROS related setup.bash files again and run roslaunch habitat_interface default.launch
to convert all habitat sensor messages into ROS messages (e.g. numpy image to ROS image). This launch file also launches a joystick controller to control the habitat agent along with visualization tools such as image view
Launch files for running hector_mapping (hector_map.launch) and navigation (move_base.launch) are also included. First run default.launch, then run either hector_map.launch or move_base.launch. Note that to run navigation, you need to change the value of the "map_path" variable in move_base.launch to point to the map you want to use. In addition, you can use the get_ros_map.py script in habitat-api/habitat_ros to generate a ROS/Rviz compatible map that you can compare/overlay with your SLAM generated map to evaluate the SLAM generated map's accuracy (you'll need to modify the origin of your map.yaml file to have the two maps line up in Rviz).
Published:
Subscribed
You can change simulator settings by modifying the hab_ros_interface.py file.
To change scenes, change the config file fed into the environment initializer in hab_ros_interface.py.
my_env = sim_env(env_config_file="path/to/your/custom/config/file.yaml")
Specifically, inside your config file, modify the DATASET tag. See screenshot below for an example
You can modify the _update_position and _update_attitude methods in the sim_env class in hab_ros_interface.py to change the robot's behaviour at each time step. (e.g. you can specify that the robot has a maximum acceleration of 0.1m/s^2)
Change the _sensor_rate class variable in hab_ros_interface.py
See habitat-api/examples/register_new_sensors_and_measures.py for details.
As an example, I added an additional sensor called BC_SENSOR (this is just another RGB sensor) and you can access the readings of this sensor if you add this sensor to your config file (see screenshot below as an example). You can obtain its values by calling self.observations['bc_sensor']
in hab_ros_interface.py
Place habitat-api in a catkin work space (i.e. make Habitat a ROS package). The advantage of wrapping Habitat in a catkin_ws is that we can use launch files to remap topics and do unit and integration tests. The main difficulty of this implementation is that most of habitat's files specify relative paths, while ROS commands such as rosrun and roslaunch sets "the working directory of all nodes to $ROS_HOME, which in most cases will be $HOME/.ros" (source: https://answers.ros.org/question/235337/unable-to-read-a-file-while-using-relative-path/)
Therefore, to wrap habitat in a catkin_package, we either need to change all paths in habitat-api (possibly also habitat-sim) to be absolute paths, or need to somehow use ROS' rospack find
feature ($find some_package
feature in ROS launch files) to change all relative paths to absolute paths at run time.
Below is an image showing a sample habitat episode specification file. You can see the path to the scene is relative to habitat-api's root directory.
Simplify the procedures to run Habitat with ROS. For example, eliminate the need to manually comment out Anaconda related lines in ~/.bashrc to run system's default ROS. Currently I'm searching/developing a robust method to run ROS in a python2.7 Anaconda environment.
agentRadius
value should change the dimension of the agentorigin
value in the SLAM generated map's .yaml file. In the future, this step could possibly be automated by some vision/optimization technique that shifts one map's origin. Additionally, by understanding how SLAM packages like hector_slam creates its map.pgm and map.yaml, we can do things like crop the map.pgm picture in a way such that only relevant pixels remain (grey pixels surrounding the map are cropped away). This allows us to potentially select the bottom left corner of both the SLAM generated and ground truth map to be the map origin and overlay the two maps. X
m/s^2. This can be done by modifying the _update_position and _update_attitude methods in the sim_env class in hab_ros_interface.py When adding new features, please note that in the habitat environment, the agent looks along the -z axis,and to the right of where the agent looks is the -x axis. Above the agent is the +y axis