search

facebookresearch / rlmeta

RLMeta is a light-weight flexible framework for Distributed Reinforcement Learning Research.
MIT License
284 stars 28 forks source link

Longer-term and relation to other RL libraries under Meta #33

Closed etaoxing closed 2 years ago

etaoxing commented 2 years ago

Hi, excited to see this work on distributed RL, building off moolib (and TorchBeast originally). I'm wondering what the longer-term direction of this project is?

Will functionality be merged into TorchRL (which mentions an upcoming IMPALA implementation)? https://github.com/facebookresearch/rl#upcoming-features

Is moolib still being maintained? https://github.com/facebookresearch/moolib/issues/32#issuecomment-1085730793

There are so many RL libraries these days.

xiaomengy commented 2 years ago

Hi, thanks for the interest. Let me provide some of my personal vision of these libs/frameworks.

  1. Moolib is mainly for providing the low-level functionality for distributed ML trainings, not only RL.
  2. RLMeta is a distributed RL framework which is based on moolib and provide high-level distributed RL algorithms for RL search.
  3. TorchRL is mainly a RL lib instead of a framework. The main purpose of TorchRL is to provide official PyTorch implementation for RL components which can be used for other RL frameworks. And because of the timelines and priorities of different teams, TorchRL will be moved to pytorch repo later. Also TorchRL actually merged some of the works in RLMeta. For example the replay buffer is moved from RLMeta. And later we will consider to let RLMeta use TorchRL components.

This is a general high-level vision of the things. We don't want to put all of the things in a single repo but have different repos for different use cases. But to be honest we cannot predict the future, so this is a personal view of the current structure.

Hopefully this can help clarify some of your questions.

etaoxing commented 2 years ago

Thanks for the insight, looking forward to seeing the development progress! High-quality and easy to use distributed RL code is hard to come by.