:cherries: Cherry in PyTorch

Reinforcement Learning & other bedtime stories in PyTorch. This repository is pytorch implementation of the Intro RL course & Advanced RL course. The name cherry is derived from Yann Lecun's statement regarding RL being the cherry on the ML cake. For training/playing the agent within conda env, please install the external dependencies. Not to be confused with a package of similar name.

Installation
- Conda or Docker
Sample usage
- Cartpole-v0
Dependencies (Conda only)
- Vizdoom, ALE, Baselines
Architectures & Agents
Environments
Design principle
Reproducibility

:computer: Installation

Tested with ubuntu:18.04 only

:snake: Conda

# After installed external dependencies
conda create -n cherry python=3.6
conda activate cherry
pip install -r requirements.txt
pip install -e .

:whale: Docker

# CPU
docker pull moabitcoin/cherry-pytorch-cpu
docker run moabitcoin/cherry-pytorch-cpu cherry --help
# GPU
docker pull moabitcoin/cherry-pytorch-gpu
docker run moabitcoin/cherry-pytorch-gpu cherry --help

:whale: Docker (build from scratch)

# CPU
make install dockerfile=Dockerfile.cpu dockerimage=moabitcoin/cherry-pytorch-cpu
make run dockerimage=moabitcoin/cherry-pytorch-cpu
# GPU
make install dockerfile=Dockerfile.gpu dockerimage=moabitcoin/cherry-pytorch-gpu
make gpu dockerimage=moabitcoin/cherry-pytorch-gpu

:tada: Usage

# Inside the virtual environment / Docker
cherry --help
cherry train --help

usage: cherry train [-h] -c CONFIG_FILE [-d {gpu,cpu}] [-l {info,debug}]

optional arguments:
  -h, --help            show this help message and exit
  -c CONFIG_FILE, --config_file CONFIG_FILE
                        Path to Config file (default: None)
  -d {gpu,cpu}          Device to run the train/test (default: gpu)
  -l {info,debug}, --log {info,debug}
                        Set verbosity for the logger (default: info)

:eyes: Example (Cartpole)

Train

cherry train -c configs/control.yaml -d cpu

Play

# <model_dest> in configs/control.yaml
cherry play -c <model_dest>/control-<commit-gitsha>.yaml -d cpu -m <model_dest>/agent-final-<commit-gitsha>.pth

Visualise

# <state_dest> in configs/control.yaml
vlc <state_dest>

Architectures

We support 4 feedforward architectures within cherry. We plan to expand the list of architectures to include Recurrent/Transformer/Memory architectures. If your personal model flavour is missing, please open an issue with links to architecture details.

Agents

We support 4 Agents within cherry. We plan to expand the list of agents to include TRPO/PPO. Please feel free to make an agent request by opening an issue with useful links to publication(s)/existing implementation.

Environments

We support 4 environments within cherry. This list is planned to be expanded to include Robotics and other 3rd party environments.

:two_men_holding_hands: Dependencies

:godmode: ViZDoom

Install dependencies as outlined here

sudo apt-get install libcanberra-gtk-module
pip install vizdoom
# If on a headless machine
export DISPLAY=:0

:space_invader: Arcade Learning Environment

https://github.com/openai/gym/blob/master/docs/environments.md#atari

:chart_with_upwards_trend: OpenAI RL baselines

Needed for Deepmind styled pre-processing wrappers. Needs TF>=1.14 :woman_facepalming:

pip install tensorflow_gpu==1.14
https://github.com/openai/baselines/tree/master#installation

:tv: Remote Render

https://github.com/openai/gym/tree/0.15.4#rendering-on-a-server
xvfb-run -s "-screen 0 1400x900x24" bash

moabitcoin / cherry-pytorch

readme