HorizonRobotics / SocialRobot

Apache License 2.0
71 stars 20 forks source link

Social Robot

A python environment for developing interactive learning agent with language communication ability.

Install dependency

SocialBot is built on top of Gazebo simulator. You need to install Gazebo first using the following command:

curl -sSL http://get.gazebosim.org | sh

If you already have a gazebo in your system, please make sure its version is greater than 9.6. You can check gazebo version by running gazebo --version. SocialRobot had been tested with Gazebo 9.6 and Gazebo 10.0.

You might need to add the model path in this repo to GAZEBO_MODEL_PATH:

export GAZEBO_MODEL_PATH=REPO_ROOT/python/social_bot/models:$GAZEBO_MODEL_PATH

You also need to install the following packages:

apt install python3-tk

To compile

cd REPO_ROOT
git submodule update --init --recursive
mkdir build
cd build
cmake ..
make -j
cd REPO_ROOT
pip3 install -e .

if cmake .. complains about cannot find configuration file provided by "gazebo", run sudo apt install libgazebo9-dev where the version number 9 comes from gazebo --version

To run test

Play Ground Test

python3 python/social_bot/envs/play_ground.py

Simple Navigation Test

cd REPO_ROOT/examples
python3 test_simple_navigation.py

To see the graphics, you can open another terminal and run

GAZEBO_MODEL_PATH=`pwd`/../python/social_bot/models gzclient

Environments and Tasks

We provide OpenAI gym interfaces to easily apply different RL algorithms into these different environments. The registered names for them are listed as below:

SocialBot-PlayGround-v0
SocialBot-SimpleNavigation-v0
SocialBot-SimpleNavigationDiscreteAction-v0
SocialBot-SimpleNavigationLanguage-v0
SocialBot-SimpleNavigationSelfStatesLanguage-v0
SocialBot-Pr2Gripper-v0
SocialBot-ICubWalk-v0
SocialBot-ICubWalkPID-v0

Some environments support the teacher-student learning procedure, in which the task is defined by the teacher, and an interaction with the teacher via a sentence is performed during each environment step. You could enable the procedure by using the environment whose name contains "language" or through gin configuration.

Play Ground

A flexiable environment in which worldfile, tasks and agent are all configurable through gin file. The default configuration for play ground is goal task and pioneer agent. You can choose the agent type in the environment by setting the paramenter "agent_type". We support pioneer2dx, pr2, turtlebot, kuka youbot, kuka LWR4 and icub for now. ICub is an humanoid robot meant for more complex tasks in the future. You could also choose icub_with_hands, which is a more advanced version of icub equipped with 2 cameras and dexterous hands. The following are images of these 5 robots:

pioneer pr2 turtlebot icub youbot

The common functions for agents like setting observation space and action space are in file gazebo_agent.py. Class GazeboAgent sets those things up according to agent configuration file agent_cfg.json.

Your can configure the type of observation using gin: with_language or not, use_image or not, and image_with_internal_states or not. Possible situations can be: low-dimensional full states, low-dimensional full states with language sentence, image from the camera of agent, image with internal states, image with language sentence, image with both internal states and language sentence. By default low-dimensional full states is used.

The action specification is defined in the joint list of file agent_cfg.json.

Tasks are in file tasks.py. To implement a new task, you should:

  1. Inherit tasks.Task, speficy the compatible agent types in "Task.compatible_agents".
  2. Override Task.__init__() to set things up.
  3. Override Task.run() to yeild reward, observation, and optional language string at each time step.
  4. Override Task.task_specific_observation() to speficy the observation for the task for the non-image case.

The tasks we have for now are listed as below:

Simple Navigation

A simple navigation task for a pioneer2dx agent with camera image as observation. This environment is only compatible with goal task and agent pioneer2dx.

PR2 Gripping

iCub Walking

Training Examples with Agent Learning Framework (Alf)

Train goal_task task with Alf actor-critic

cd REPO_ROOT/examples/
python -m alf.bin.train --root_dir=~/tmp/goal_task --gin_file=goaltask_sac_pioneer.gin --alsologtostderr

To play:

python -m alf.bin.play --root_dir=~/tmp/goal_task --gin_file=goaltask_sac_pioneer.gin --num_episodes=20

The playing back can also be recorded to a video file like this:

python -m alf.bin.play --root_dir=~/tmp/goal_task --gin_file=goaltask_sac_pioneer.gin --record_file=goal_task.mp4

You can find the gin config files for other tasks here. Training curves for the tasks are in folder here.

Demonstrating using keyboard and mouse

We have a keyboard interface for the tasks, which can be used to generate demonstrations, potentially can be used for teacher-student learning formula or imitation learning. Try this for a test:

cd REPO_ROOT/python/social_bot/
python keybo_control.py

Move the agent around by key "W, A, S, D", open or close gripper by key "E", and move the robot arm (if there is, such as agent youbot) by mouse for lef-right, forward-backward direction, and key "R, F" for up-down. Note that some tricks are used to make the operating more friendly. So you are not controlling the raw joints as the algorthim. It's not so easy, for some task you may need some trials to complete it. An example:

stack

Trouble shooting

python

You need to make sure the python you use matches the python found by cmake. You can check this by looking at REPO_ROOT/build/CMakeCache.txt

display

You need to make sure your DISPLAY environment variable points to a valid display, otherwise camera sensor cannot be created and there will be the following error:

[Err] [CameraSensor.cc:112] Unable to create CameraSensor. Rendering is disabled.
...
gazebo::rendering::Camera*]: Assertion `px != 0' failed.

You can find out the correct value for DISPLAY envorinment variable by running echo $DISPLAY in a terminal opened from the desktop. You can verify whether the DISPLAY is set correctly by running gazebo