When using the SimpleReplayBuffer, the sampling performance depends on the number of samples due to deque. This is caused by the slow indexed access to elements in the middle of the collection.
The simple switch to a list improves the performance significantly. An example for SAC on the Walker stand task from the DeepMind Control Suite with a buffer size of 1M is given below.
KristianHolsheimer
commented
2 years ago
When using the
SimpleReplayBuffer, the sampling performance depends on the number of samples due todeque. This is caused by the slow indexed access to elements in the middle of the collection.The simple switch to a
listimproves the performance significantly. An example for SAC on the Walker stand task from the DeepMind Control Suite with a buffer size of 1M is given below.