Benchmark downloader and processor

[TimotheeMathieu]

It would be nice to have a function to fetch and process benchmark results from various places like stablebaselines3-zoo or cleanrl, so they can be used easily to compare with the results of a rlberry user.

[TimotheeMathieu]

Google Atari bucket: https://console.cloud.google.com/storage/brow ser/rl-benchmark-data/ALE SB3: https://github.com/DLR-RM/rl-baselines3-zoo/tree/master/logs/benchmark WARNING only one seed cleanrl: https://wandb.ai/openrlbenchmark/openrlbenchmark/reportlist (no idea what data they have)

There are other.

Goal is to fit entry format for comparator (https://rlberry-py.github.io/rlberry/basics/comparison.html).

@YannBerthelot , @KohlerHECTOR, @mmcenta, @AleShi94 do you have any other idea of benchmark that we could use ?

[JulienT01]

link to Google Atari bucket: https://console.cloud.google.com/storage/browser/rl-benchmark-data/atari_100k associated colab : https://colab.research.google.com/drive/1a0pSD-1tWhMmeJeeoyZM1A-HCW3yf1xR?usp=sharing#scrollTo=eV8ci5frgPFO associated paper : https://arxiv.org/pdf/2108.13264.pdf

[TimotheeMathieu]

There is also https://github.com/DLR-RM/rl-trained-agents/ And maybe we can also have importer for HuggingFace

[YannBerthelot]

Just came out: https://arxiv.org/abs/2402.03046, associated github : https://github.com/openrlbenchmark/openrlbenchmark

Le mer. 7 févr. 2024 à 09:44, TimotheeMathieu @.***> a écrit :

There is also https://github.com/DLR-RM/rl-trained-agents/

— Reply to this email directly, view it on GitHub https://github.com/rlberry-py/rlberry/issues/395#issuecomment-1931542483, or unsubscribe https://github.com/notifications/unsubscribe-auth/ALWSWPWWFAETWHGN4GSTPGDYSM5FTAVCNFSM6AAAAAA752V552VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMZRGU2DENBYGM . You are receiving this because you were mentioned.Message ID: @.***>

[KohlerHECTOR]

La classe @YannBerthelot !!! Viens tu présentes aujourd'hui ?