search

CAS-CLab / Training-Tricks-for-Binarized-Neural-Networks

The collection of training tricks of binarized neural networks.
BSD 2-Clause "Simplified" License
71 stars 11 forks source link

Question about 11th - Pooling block order #1

Closed A-suozhang closed 4 years ago

A-suozhang commented 4 years ago

Thank you for your great work, it really helped me a lot. However, I'm a bit confused about the 11th tip, which reorders the "conv-bn-relu-pooling" to "conv-polling-relu-bn". Is there a paper reference for this method, or other insight about why this would work?

HolmesShuan commented 4 years ago

Thanks for your interest. Conv-BN-ReLU-Pooling, Conv-Pooling-ReLU-BN and Conv-Pooling-BN-ReLU mostly come from our practical experience. It is hard to justify which one is better than the others. It depends on your tasks and backbone networks.

The reference can be found in "XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks" section 4.3 "Ablations Studies".