search

rlworkgroup / garage

A toolkit for reproducible reinforcement learning research.
MIT License
1.87k stars 310 forks source link

RLlib vs Garage (RLLab) #1129

Closed MadcowD closed 4 years ago

MadcowD commented 4 years ago

Hi there,

I was wondering if anyone had established a reasonable comparison to rllib (ray). I am trying to chose between this and that and am having trouble making that decision.

ryanjulian commented 4 years ago

@MadcowD here are some quick "this vs that" comparisons

Garage rllib
Focuses on single-machine environments for now, but allows you to add distributed components like sampling Focuses aggressively on cluster environments, requiring you design your algorithm for clusters from the start
Community-supported Supported by Berkeley RISE lab and a new start-up called Anyscale
Stand-alone project Technology demonstration project for the larger ray distributed computing library
No A2C family algorithms yet Has A2C family algorithms
Fewer, simpler abstractions More, more complex abstractions (builders, mixins, hooks, trainers, etc.)
No built-in hyperparam tuning interface Built-in support for "ray tune" tool
Generally easier to override and modify Generally harder to override and modify (according to reports I've received)
Supports both PyTorch and TensorFlow Supports both PyTorch and TensorFlow
No (explicit) multi-agent support yet Multi-agent support
Multi-task and meta-RL support No multi-task and meta-RL support
MadcowD commented 4 years ago

Thank you!!!!!

MadcowD commented 4 years ago

@ryanjulian can I ask what the barriers to A2C style algorithms are?

ryanjulian commented 4 years ago

No technical barriers, just time. We spend a lot of time focusing on RL algorithms for continuous control, and A2C-style algorithms are more useful for gameplay domains. They would not be difficult to implement in garage, especially because we already have pixel-DQN.

aruhela commented 2 years ago

Is it possible to train Garage DQN algorithm with multiple nodes using CPUs only? If yes, can you share a demo script or program. My sampling works at 2 nodes, 56 PPN using Ray Sampler.