search

wookayin / expt

Experiment. Plot. Tabulate.
MIT License
67 stars 5 forks source link

Open RL Benchmark Integration #3

Open vwxyzjn opened 2 years ago

vwxyzjn commented 2 years ago

Hi Jongwook,

Per our earlier discussion, here is the repository for Open RL Benchmark https://github.com/openrlbenchmark/openrlbenchmark.

We now have tracked experiments from a pretty impressive collection of RL libraries such as CleanRL, SB3, Tianshou, sfujim/TD3, and jaxrl at https://wandb.ai/openrlbenchmark/openrlbenchmark/reportlist.

Would it be possible for us to directly plot charts from https://wandb.ai/openrlbenchmark/openrlbenchmark/reports/MuJoCo-CleanRL-s-TD3--VmlldzoxNjk4Mzk5 using expt as an example?

Thanks!

wookayin commented 2 years ago

Hi Costa, this is really a great idea and thanks for your efforts on maintaining the benchmark results. I'd like to use the cleanRL benchmarks data for the example notebook and/or documentation, replacing the current OpenAI's example data. Please stay tuned!

vwxyzjn commented 2 years ago

Thank you @wookayin. I have been playing with the plotting API and think it's well-designed and easy to use. I made some prototypes here https://github.com/openrlbenchmark/openrlbenchmark/pull/2.

image

Totally understand you are busy :) and thanks for the response!