search

Kaixhin / Rainbow

Rainbow: Combining Improvements in Deep Reinforcement Learning
MIT License
1.56k stars 282 forks source link

IndexError: index 262141 is out of bounds for axis 0 with size 231071 #76

Closed lucifer2859 closed 3 years ago

lucifer2859 commented 3 years ago

· File "/home/rlcube/dchen/Rainbow/memory.py", line 148, in sample │································· probs, idxs, tree_idxs, states, actions, returns, next_states, nonterminals = self._get│································· _samples_from_segments(batch_size, p_total) # Get batch of valid samples │································· File "/home/rlcube/dchen/Rainbow/memory.py", line 128, in _get_samples_from_segments │································· probs, idxs, tree_idxs = self.transitions.find(samples) # Retrieve samples from tree w│································· ith un-normalised probability │································· File "/home/rlcube/dchen/Rainbow/memory.py", line 75, in find │································· indices = self._retrieve(np.zeros(values.shape, dtype=np.int32), values) │································· File "/home/rlcube/dchen/Rainbow/memory.py", line 71, in _retrieve │································· return self._retrieve(successor_indices, successor_values) │································· File "/home/rlcube/dchen/Rainbow/memory.py", line 71, in _retrieve │································· return self._retrieve(successor_indices, successor_values) │································· File "/home/rlcube/dchen/Rainbow/memory.py", line 71, in _retrieve │································· return self._retrieve(successor_indices, successor_values) │································· [Previous line repeated 13 more times] │································· File "/home/rlcube/dchen/Rainbow/memory.py", line 67, in _retrieve │································· left_children_values = self.sum_tree[children_indices[0]] │································· IndexError: index 262141 is out of bounds for axis 0 with size 231071

Kaixhin commented 3 years ago

This is probably a duplicate of https://github.com/Kaixhin/Rainbow/issues/41 and you will need to set --priority-exponent 0.

chshong commented 3 years ago

I have met the same issue several times and I think the issue is due to Lines 66-68 in memory.py

if children_indices[0, 0] >= self.sum_tree.shape[0]: return indices left_children_values = self.sum_tree[children_indices[0]]

Since the if statement only checks children_indices[0,0], remaining (batch_size - 1) indices are not checked against the max size of the sum tree, ending up in IndexError in some occasions