[Bug]: Potential bug in TorchRunningMeanStd

RLE-Foundation / rllte

Long-Term Evolution Project of Reinforcement Learning

https://docs.rllte.dev/

MIT License

453 stars 84 forks source link

Open dominikonysz opened 4 months ago

dominikonysz commented 4 months ago

During my debugging I came across the computation of the updated value for the running mean:

It doesn't really influence me as I'm not using it right now but I feel like it should instead be:

new_mean = mean + delta * batch_count / tot_count

yuanmingqi commented 4 months ago

@roger-creus Hi Roger, can you check this issue?

yuanmingqi commented 4 months ago

@dominikonysz We have uploaded the update and marked you as a co-author. Thank you for the issue.