Fix dqn model evals

[sdpkjc]

Description

Fixes #380

Types of changes

• [x] Bug fix
• [ ] New feature
• [ ] New algorithm
• [ ] Documentation

Checklist:

• [x] I've read the CONTRIBUTION guide (required).
• [x] I have ensured pre-commit run --all-files passes (required).
• [x] I have updated the tests accordingly (if applicable).
• [ ] I have updated the documentation and previewed the changes via mkdocs serve.
  • [ ] I have explained note-worthy implementation details.
  • [ ] I have explained the logged metrics.
  • [ ] I have added links to the original paper and related papers.

If you need to run benchmark experiments for a performance-impacting changes:

• [ ] I have contacted @vwxyzjn to obtain access to the openrlbenchmark W&B team.
• [ ] I have used the benchmark utility to submit the tracked experiments to the openrlbenchmark/cleanrl W&B project, optionally with --capture-video.
• [ ] I have performed RLops with python -m openrlbenchmark.rlops.
  • For new feature or bug fix:
    • [ ] I have used the RLops utility to understand the performance impact of the changes and confirmed there is no regression.
  • For new algorithm:
    • [ ] I have created a table comparing my results against those from reputable sources (i.e., the original paper or other reference implementation).
  • [ ] I have added the learning curves generated by the python -m openrlbenchmark.rlops utility to the documentation.
  • [ ] I have added links to the tracked experiments in W&B, generated by python -m openrlbenchmark.rlops ....your_args... --report, to the documentation.

[vercel[bot]]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
cleanrl	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	May 6, 2023 9:56pm

[sdpkjc]

Next -> Add test cases

[sdpkjc]

Can we modify the existing test cases to test them, or create a new test file for them?

def test_dqn_jax():
    subprocess.run(
        "python cleanrl/dqn_atari_jax.py --save-model True --learning-starts 10 --total-timesteps 16 --buffer-size 10 --batch-size 4",
        shell=True,
        check=True,
    )

[sdpkjc]

The model evaluation dependency environment is related to the algorithm dependency environment. If we create a new test file for the model evaluation, it will multiply the number of test files. I suggest just adding --save-model True, or something simple way.

[vwxyzjn]

The model evaluation dependency environment is related to the algorithm dependency environment. If we create a new test file for the model evaluation, it will multiply the number of test files. I suggest just adding --save-model True, or something simple way.

This sounds good to me!

[sdpkjc]

Thanks for your review. 👌🫡