DiverseRL

DiverseRL is a repository that aims to implement and benchmark reinforcement learning algorithms.

This repo aims to implement algorithms of various sub-topics in RL (e.g. model-based RL, offline RL), in wide variety of environments.

Features

• Wandb logging
• Tensorboard

Installation

You can install the requirements by using Poetry.

git clone https://github.com/moripiri/DiverseRL.git
cd DiverseRL

poetry install

Algorithms

Currently, the following algorithms are available.

Model-free Deep RL

Model-free Deep RL algorithms are set of algorithms that can train environments with state-based observations without model.

• DQN (Deep Q-Network)
  • Double DQN
  • Dueling DQN
  • NoisyNet
• DDPG (Deep Deterministic Policy Gradients)
• TD3 (Twin Deep Delayed Deterministic Policy Gradients)
• SAC (Soft Actor Critic)
• PPO (Proximal Policy Gradients)

Pixel RL

Pixel RL contains set of algorithms that are set to train environments with high-dimensional images as observations, Such as Atari 2600 and dm-control.

• DQN (Deep Q-Network) (for Atari 2600)
• SAC (Soft Actor Critic) (for dm-control)
• PPO (Proximal Policy Gradients) (for Atari 2600)
• SAC-AE (Soft Actor Critic with Autoencoders) (for dm-control)
• CURL (Contrastive Unsupervised Representations for Reinforcement Learning) (for dm-control)
• RAD (Reinforcement Learning with Augmented Data) (for dm-control)
• DrQ (Data-Regularized Q) (for dm-control)
• DrQ-v2 (Data-Regularized Q v2) (for dm-control)

Classic RL

Classic RL algorithms that are mostly known by Sutton's Reinforcement Learning: An Introduction. Can be trained in Gymnasium's toy text environments.

• SARSA
• Q-learning
• Model-free Monte-Carlo Control
• Dyna-Q

Getting Started

Training requires two gymnasium environments(for training and evaluation), algorithm, and trainer.

from diverserl.algos import DQN
from diverserl.trainers import DeepRLTrainer
from diverserl.common.utils import make_envs

env, eval_env = make_envs(env_id='CartPole-v1')

algo = DQN(env=env, eval_env=eval_env, **config)

trainer = DeepRLTrainer(
    algo=algo,
    **config
)

trainer.run()

Or you use hydra by running run.py.

python run.py env=gym_classic_control algo=dqn trainer=deeprl_trainer algo.device=cuda trainer.max_step=10000