remove the default added epsilon in favor of capping the std (and min difference between max/min) to a epsilon value
make the normalization not in-place
add test for the bahavior of the normalization
The reasons for these proposals are as follow:
remove the default added epsilon in favor of capping the std:
in both torchvision and the original ACT code a capping of the min value of std is used instead of a constantly added epsilon. this bring us in-line with what other well-used repos are doing
in most cases std is not null we are thus closer to the theoretical behavior for the normalization
make the normalization not in-place
right now the normalization was returning a tensor but also modifying it in-place instead the function. This can be error-prone as the user typically doesn't expect a function returning a value to also modify the input in-place.
for now I decided to make it not in-place. An alternative could be to make it fully in-place and remove the returned value. Less intuitive according to me for probably minor speed/memory gains
How it was tested
Added extensive tests for the behavior in tests/test_policies.py::test_normalize
What this does
This PR work on the Normalization in ACT to:
The reasons for these proposals are as follow:
How it was tested
Added extensive tests for the behavior in
tests/test_policies.py::test_normalize
How to checkout & try? (for the reviewer)
Run the tests:
This change is