Replay buffer fix - Githubissues

Kaixhin / Rainbow

Rainbow: Combining Improvements in Deep Reinforcement Learning

MIT License

1.56k stars 282 forks source link

Replay buffer fix #79

Closed benblack769 closed 3 years ago

benblack769 commented 3 years ago

I got the following error on several different runs, making the library somewhat unusable:

  File ".../Rainbow/memory.py", line 68, in _retrieve
    left_children_values = self.sum_tree[children_indices[0]]
IndexError: index 2097149 is out of bounds for axis 0 with size 2048575

Seems to be a repeat of #76

Here is my proposed fix:

Does this seem like a valid fix?

Kaixhin commented 3 years ago

Could you add some inline comments to explain the logic?

benblack769 commented 3 years ago

Turns out it doesn't work. I'll close this PR and resubmit when something works.