[Bug]: RunningMeanStd overflowing - Githubissues

DLR-RM / stable-baselines3

PyTorch version of Stable Baselines, reliable implementations of reinforcement learning algorithms.

https://stable-baselines3.readthedocs.io

MIT License

8.35k stars 1.6k forks source link

[Bug]: RunningMeanStd overflowing #1953

Open spiglerg opened 1 week ago

spiglerg commented 1 week ago

🐛 Bug

RunningMeanStd is not overflow safe, and overflows when running large-scale training (e.g., on a cluster).

To Reproduce

I'm submitting a pull request with a proposal to address the problem.

Relevant log output / Error message

No response

System Info

No response

Checklist

[X] My issue does not relate to a custom gym environment. (Use the custom gym env template instead)
[X] I have checked that there is no similar issue in the repo
[X] I have read the documentation
[X] I have provided a minimal and working example to reproduce the bug
[X] I've used the markdown code blocks for both code and stack traces.

araffin commented 1 day ago

I have provided a minimal and working example to reproduce the bug

hello please do not forget this step, that would allow us better to understand the problem and when it occurs.