Fix/dreamer v3 - Githubissues

Eclectic-Sheep / sheeprl

Distributed Reinforcement Learning accelerated by Lightning Fabric

https://eclecticsheep.ai

Apache License 2.0

274 stars 26 forks source link

Fix/dreamer v3 #253

Closed michele-milesi closed 3 months ago

michele-milesi commented 3 months ago

Summary

Describe the purpose of the pull request, including:

Added LayerNormFP32 and LayerNormChannelLastFP32 for computing the layer norm always in full precision.
Added possibility to select, for each model, which layer norm to use in Dreamer V3
Added possibility to specify the kwargs of the LN layers for each model in Dv3
Added learnable initial recurrent state in dv3
Moments in dv3 are always computed in full precision
Fixed env interaction in dreamer and p2e algorithms

Type of Change

Please select the one relevant option below:

Bug fix (non-breaking change that solves an issue)

Checklist

Please confirm that the following tasks have been completed:

[x] I have tested my changes locally and they work as expected. (Please describe the tests you performed.)
[x] I have added unit tests for my changes, or updated existing tests if necessary.
[x] I have updated the documentation, if applicable.
[x] I have installed pre-commit and run locally for my code changes.

Screenshots or Visuals (Optional)

If applicable, please provide screenshots, diagrams, graphs, or videos of the changes, features or the error.

ww-final

The reward obtained by DreamerV3 in the Walker Walk environment from DMC with the modifications described above.