search

itayhubara / BinaryNet.tf

BNN implementation in tensorflow
165 stars 54 forks source link

Test accuracy changes based on batch size #9

Open jonkoi opened 6 years ago

jonkoi commented 6 years ago

Hi,

I trained BNN_cifar10 with batch size 256 and got the accuracy of 91.2% (higher than baseline cifar10) . However, when I use the evaluation script alone to test the model. I noticed that both the confidence and the test accuracy worsens as the test batch size get smaller, which should not be the case. The extreme case is batch size of 1. In this case, the model will only have 1 output. When I look at the output for each image (batch size 1), they are barely different. I checked if the input is different because of batch size but they are the same.

Do you know what can cause this?

mikuhatsune commented 6 years ago

I think BatchNorm is handled incorrectly during evaluation...