This PR introduces the replay_ratio as specified in #223. In particular we have removed both the per_rank_gradient_steps and train_every, since those two can be computed with the single output from the newly introduced Ratio class. We have also modified how the Dreamer-V1 family algos compute the exploration actions, while removing that from Dreamer-V2 and Dreamer-V3 since those are unused by those algos.
Type of Change
Please select the one relevant option below:
New feature (non-breaking change that adds functionality)
Checklist
Please confirm that the following tasks have been completed:
[x] I have tested my changes locally and they work as expected. (Please describe the tests you performed.)
[x] I have added unit tests for my changes, or updated existing tests if necessary.
[x] I have updated the documentation, if applicable.
[x] I have installed pre-commit and run locally for my code changes.
Screenshots or Visuals (Optional)
If applicable, please provide screenshots, diagrams, graphs, or videos of the changes, features or the error.
Additional Information (Optional)
Please provide any additional information that may be useful for the reviewer, such as:
Any potential risks or challenges associated with the changes.
Any instructions for testing or running the code.
Any other relevant information.
Thank you for your contribution! Once you have filled out this template, please ensure that you have assigned the appropriate reviewers and that all tests have passed.
Summary
This PR introduces the
replay_ratioas specified in #223. In particular we have removed both theper_rank_gradient_stepsandtrain_every, since those two can be computed with the single output from the newly introducedRatioclass. We have also modified how the Dreamer-V1 family algos compute the exploration actions, while removing that from Dreamer-V2 and Dreamer-V3 since those are unused by those algos.Type of Change
Please select the one relevant option below:
Checklist
Please confirm that the following tasks have been completed:
Screenshots or Visuals (Optional)
If applicable, please provide screenshots, diagrams, graphs, or videos of the changes, features or the error.
Additional Information (Optional)
Please provide any additional information that may be useful for the reviewer, such as:
Thank you for your contribution! Once you have filled out this template, please ensure that you have assigned the appropriate reviewers and that all tests have passed.