Update docstring of normalize reward

[ffelten]

Description

As discussed in https://discord.com/channels/961771112864313344/1243134869559578675/1270855568336359435 I updated the docstring to reflect the fact that it does not normalize.

Fixes # (issue)

Type of change

Please delete options that are not relevant.

• [x] Documentation only change (no code changed)
• [ ] Bug fix (non-breaking change which fixes an issue)
• [ ] New feature (non-breaking change which adds functionality)
• [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
• [ ] This change requires a documentation update

Screenshots

Please attach before and after screenshots of the change if applicable.

Checklist:

• [x] I have run the pre-commit checks with pre-commit run --all-files (see CONTRIBUTING.md instructions to set it up)
• [x] I have commented my code, particularly in hard-to-understand areas
• [x] I have made corresponding changes to the documentation
• [x] My changes generate no new warnings
• [ ] I have added tests that prove my fix is effective or that my feature works
• [ ] New and existing unit tests pass locally with my changes

[ffelten]

Could you update the top sentence and the note to include a discussion that it is normalizing the discounted future rewards for an episode.

Not sure to understand what you want in the end haha

Would you be interested in adding a feature that the reward is actually normalized?

How can you normalize without knowing the reward bounds?

[pseudo-rnd-thoughts]

The critical feature of NormalizeReward is that it doesn't normalize rewards to have a mean of 0 and std of 1 but that the discounted rewards has a mean of 0 and std of 1. Could the note certain explain this and docstring only that it normalizes the sum of discounted rewards

How can you normalize without knowing the reward bounds?

Like the observation normalization, you use a running mean and standard deviation to normalize. So this evolves over time as different environment rewards are visited but does the expect case of normalize the individual rewards not cumulative rewards.