search

Kaixhin / Rainbow

Rainbow: Combining Improvements in Deep Reinforcement Learning
MIT License
1.56k stars 282 forks source link

Fix IndexError caused by Numpy rounding #81

Closed Aladoro closed 3 years ago

Aladoro commented 3 years ago

Small fix for the IndexError from #76.

Particularly, this is caused by Numpy's rounding approximations in the computations for the SegmentTree total, making it (very slightly) inexact. Hence, in the event that we have some small overshooting it is possible (yet, quite unlikely) that we sample a SegmentTree value beyond the actual total value of the tree elements, causing the index to overshoot the actual size of _sumtree.

Constraining the children leaf node indexes to be within the SegmentTree length avoids this issue, adding an irrelevantly small positive bias to sampling the last element in the tree.

Kaixhin commented 3 years ago

Thanks for this. Can you add brief comments (along the lines of what I've done so far) to the if and elif lines to explain the logic behind these?

Aladoro commented 3 years ago

Done. I hope they appropriately explain the logic. I also fixed a small typo in a comment from main.py.

By the way, thank you for maintaining this! ^^