Prototype RLops Utility

[vwxyzjn]

Description

Related to #296. To get started, run the following command

python rlops.py --exp-name ddpg_continuous_action_jax \
    --wandb-project-name cleanrl \
    --wandb-entity openrlbenchmark \
    --tags  v1.0.0b2-9-g4605546 latest \
    --env-ids Hopper-v2 Walker2d-v2 HalfCheetah-v2 \
    --output-filename compare.png \
    --report

This PR generates this wandb report (which does not display properly due to https://github.com/wandb/wandb/issues/4421; update: a temporary fix pip install git+https://github.com/wandb/wandb.git@6ccd140dabb0af529302b07f4d7ecc1235773900 works (https://github.com/wandb/wandb/tree/andrew/report-api-fixes)) and the following image

As part of this effort, I have labeled all of the existing runs in openrlbenchmark with rlops-pilot if the do not already have a label.

Types of changes

• [ ] Bug fix
• [x] New feature
• [ ] New algorithm
• [ ] Documentation

Checklist:

• [ ] I've read the CONTRIBUTION guide (required).
• [ ] I have ensured pre-commit run --all-files passes (required).
• [ ] I have updated the documentation and previewed the changes via mkdocs serve.
• [ ] I have updated the tests accordingly (if applicable).

[vercel[bot]]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Updated
cleanrl	❌ Failed (Inspect)		Dec 15, 2022 at 3:07AM (UTC)

[vwxyzjn]

Support more advanced filter syntax

python -m cleanrl_utils.rlops --wandb-project-name cleanrl \
    --wandb-entity openrlbenchmark \
    --filters 'ddpg_continuous_action_jax?user=joaogui1&tag=rlops-pilot' 'ddpg_continuous_action_jax?user=joaogui1&tag=pr-298' \
        'ddpg_continuous_action_jax?user=costa-huang&tag=rlops-pilot' 'ddpg_continuous_action_jax?user=costa-huang&tag=pr-298' \
        'ddpg_continuous_action?user=costa-huang&tag=pr-299' 'ppo_continuous_action?user=costa-huang&tag=rlops-pilot'  \
    --env-ids Hopper-v2 Walker2d-v2 HalfCheetah-v2 \
    --output-filename compare.png --report

which produces the following image and a wandb report with the same line color, thanks to @andrewtruong's amazing PR pip install git+https://github.com/wandb/wandb.git@6ccd140dabb0af529302b07f4d7ecc1235773900

[vwxyzjn]

Closed by #368