search

ray-project / ray

Ray is a unified framework for scaling AI and Python applications. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.
https://ray.io
Apache License 2.0
33.11k stars 5.6k forks source link

[rllib] Asynchronous Performance Improvements Inspired by Sample Factory [ICML 2020] #9360

Closed juliusfrost closed 1 year ago

juliusfrost commented 4 years ago

Describe your feature request

Hi, I recently came across Sample Factory: Egocentric 3D Control from Pixels at 100000 FPS with Asynchronous Reinforcement Learning at ICML 2020 which beats the rllib implementation of IMPALA. I was wondering if we could add support for the performance improvements described in their implementation. Their code is publicly available: https://github.com/alex-petrenko/sample-factory

image

image

image

ericl commented 4 years ago

Integrating a high performance sampler certainly makes sense. Maybe we can leverage sample factory directly in rllib, if the data formats are compatible. Cc @sven1977

rkooo567 commented 4 years ago

cc @sven1977

stale[bot] commented 3 years ago

Hi, I'm a bot from the Ray team :)

To help human contributors to focus on more relevant issues, I will automatically add the stale label to issues that have had no activity for more than 4 months.

If there is no further activity in the 14 days, the issue will be closed!

You can always ask for help on our discussion forum or Ray's public slack channel.

juliusfrost commented 3 years ago

This feature would still be great to have. I'm willing to work on it if I had the time.

rkooo567 commented 3 years ago

Nice! I will mark it as p2 (then it won’t be marked as stale in any case). Cc @sven1977

Rohan138 commented 1 year ago

Closing since we now have an optimized version of APPO (similar to SampleFactory) based on the IMPACT paper: https://arxiv.org/abs/1912.00167