search

wpumacay / DeeprlND-projects

Projects from the DeeprlND from Udacity
5 stars 1 forks source link

Priority based replay buffer not sampling properly #2

Closed wpumacay closed 5 years ago

wpumacay commented 5 years ago

The prioritized replay buffer is sampling just a set of experiences with very high probability over and over, in a window of samples. So far, the bellman errors seem ok, the modified agent pipeline also. The issue should be then with the priority buffer only. This might happen when assigning the priorities (on updates and additions)

wpumacay commented 5 years ago

Fixed by 9cddfe4: Modifies the SegmentedTree such that it does not use ints, but floats (it was really silly that the buffers were initialized to np.ndarrays with type int from the neutral element)