search

PierreExeter / cartpole3d_ros

The CartPole3D environment solved using reinforcement learning.
1 stars 1 forks source link

readme

CartPole3d

The Cart Pole problem is solved in 3D using Reinforcement Learning (RL). The cartpole3d environment is implemented in ROS with the package openai_ros. The RL algorithms used for training are DQN, A2C, ACKTR, PPO, TRPO implemented in the stable-baselines library.

Installation

  1. Install ROS Melodic

  2. Install the openai_ros package.

    cd ~/catkin_make/src/
git clone https://bitbucket.org/theconstructcore/openai_ros.git
cd ~/catkin_make
catkin_make
source devel/setup.bash
rosdep install openai_ros

  3. Install Gym

    pip install gym

  4. Install stable-baselines

  5. Install cartpole3d

    cd ~/catkin_make/src/
git clone https://github.com/PierreExeter/cartpole3d_ros.git
cd ~/catkin_make
catkin_make

  6. Make the scripts executable

    chmod +x ~/catkin_make/src/cartpole3d/scripts/cartpole3d_random.py
chmod +x ~/catkin_make/src/cartpole3d/scripts/cartpole3d_dqn.py
chmod +x ~/catkin_make/src/cartpole3d/scripts/cartpole3d_trpo.py
chmod +x ~/catkin_make/src/cartpole3d/scripts/cartpole3d_train_all.py
chmod +x ~/catkin_make/src/cartpole3d/scripts/cartpole3d_enjoy_all.py

  7. Create simulation workspace

    mkdir -p ~/simluation_ws/src
cd ..
catkin_make
source devel/setup.bash
rospack profile

Test the installation

roslaunch cartpole3d start_training_cartpole3d_random.launch

You should see the cartpole executing random actions in Gazebo. cartpole

Perform training

roslaunch cartpole3d start_training_cartpole3d_train_all.launch

Visualise training metrics in Tensorboard

cd ~/catkin_make/src/cartpole3d/results/
tensorboard --logdir=A2C:tensorboard_logs/A2C/,ACKTR:tensorboard_logs/ACKTR/,PPO2:tensorboard_logs/PPO2/,TRPO:tensorboard_logs/TRPO/

Observe trained agents

roslaunch cartpole3d start_training_cartpole3d_enjoy_all.launch

Plot the results

cd ~/catkin_make/src/cartpole3d/scripts/
python plot_reward.py

Supported systems

Tested with: