search

Kaixhin / Rainbow

Rainbow: Combining Improvements in Deep Reinforcement Learning
MIT License
1.56k stars 282 forks source link

Valid or Same padding in Conv2D? #34

Closed marintoro closed 5 years ago

marintoro commented 5 years ago

Hello,

I just realize by looking at the code of the open-source Rainbow agent from Google (here) that the network they use is using "Same" padding (the default in Tensorflow) while in torch the default is to use "Valid" padding.

This result in a really different network, indeed in the Tensorflow case, the first fully connected is taking a 7744 size vector (after flatten, it's [11,11,64] before flatten) while in your case using PyTorch the same fully connected is taking a 3136 size vector (it's [7,7,64] before flatten).

Don't know if it's really a problem but I thought it was interesting to notice...

Kaixhin commented 5 years ago

Dopamine was created by Brain, not DeepMind, so I think they got that detail wrong (if they were trying to replicate the original Nature network). Source code for the Nature network is here, and by default Torch7's 2D convolutions have no padding.