Why use a small batch size?

[shengchao-y]

Hi, May I ask why you have used such a small batch size? Since you have mentioned in the paper that a larger batch size would lead to a significant speed up. Why still 32 in the standard implementation? I am trying to implement your work but found it takes over a week to train one agent with the hyper parameters in the paper. Thus I am confused and curious.

[agarwl]

Good question -- The original experimentation code was based on dopamine code for online RL on Atari 2600 games which typically used a small batch size of 32. To keep things simple, we didn't change any of the parameters for the online DQN agent and stick to the batch size of 32. However, there's no reason to do so.

For RL Unplugged, we used a batch size of 256 for training on the Atari datasets. Also, recently these datasets are released in tfds at and can be used with any batch size.

[shengchao-y]

Good question -- The original experimentation code was based on dopamine code for online RL on Atari 2600 games which typically used a small batch size of 32. To keep things simple, we didn't change any of the parameters for the online DQN agent and stick to the batch size of 32. However, there's no reason to do so.

For RL Unplugged, we used a batch size of 256 for training on the Atari datasets. Also, recently these datasets are released in tfds at and can be used with any batch size.

Thanks for the answer and your nice work!