search

allenai / XNOR-Net

ImageNet classification using binary Convolutional Neural Networks
https://xnor.ai/
Other
856 stars 239 forks source link

BWN/XNOR SqueezeNet training #13

Open AGhiuta opened 7 years ago

AGhiuta commented 7 years ago

Hi,

I've been trying to train SqueezeNet in both configurations(bwn and xnor), but I can't get past 31% (24% respectively) top-1 accuracy (I was expecting accuracies similar to alexnet). I tried something similar to the GoogLenet variant depicted in the paper (I replaced the expand layers with straightforward convolutions with kernel sizes of 3x3, so there is no branching).

Have you tried to train this model? If positive, can you, please, tell me how did you do it?

Thank you, Alex

ping @mrastegari

Rahim16 commented 7 years ago

Hi,

I think the problem you are facing arises from that SqueezeNet can achieve 50x reduction in model size compared to AlexNet because it[AlexNet] has a lot of redundancy and also SqueezeNet's Fire modules were fine-tuned to achieve that level of accuracy with float weights. Binarization itself however results in a reduction of redundancy in AlexNet, thus further applying filter size reduction as proposed by SqueezeNet becomes too destructive.

BR, Rahim