Question: purpose of BatchNormSync

fyu / drn

Dilated Residual Networks

https://www.vis.xyz/pub/drn

BSD 3-Clause "New" or "Revised" License

1.1k stars 219 forks source link

Closed vadimkantorov closed 6 years ago

vadimkantorov commented 6 years ago

Is it supposed to better utilize multi-gpu case?

Are there any other differences from the standard BatchNorm? Thanks!

vadimkantorov commented 6 years ago

BatchNorm synchronization across multipe GPUs., right!