search

ikhlestov / vision_networks

Repo about neural networks for images handling
MIT License
264 stars 122 forks source link

batch_norm in dense_net.py #31

Closed jzhoulon closed 6 years ago

jzhoulon commented 6 years ago

Hi, we found that in dense_net.py, its batch_norm used with tf.contrib.layers.batch_norm api rather than tf.nn.fused_batch_norm api, is there any reason for specific using contrib.layers.batch_norm api? because in current intel mkldnn backend, contrib.layers has much overhead compared with nn.fused_batch_norm.

ikhlestov commented 6 years ago

Hi, that may be a case, but the main reason I've used batch norm from contrib layers is that I've implemented this network few years ago when no alternatives exist. I may try those layers later sometimes. Or you may change them and provide PR. Last but not least - I've made this code just as an export from authors caffe code to the tensorflow. And they used usual batch normalization. So maybe we should consider using at least tf.nn.batch_normalization method to be on par with the original code.