search

vlfeat / matconvnet

MatConvNet: CNNs for MATLAB
Other
1.4k stars 753 forks source link

Wrong net.normalization.border in pretrained models #296

Closed mfigurnov closed 7 years ago

mfigurnov commented 8 years ago

I've encountered a very strange behavior of cnn_imagenet_evaluate: in old versions of MatConvNet (March 2015 or so), VGG-16 model achieves 11.6% err5, while in new versions of MatConvNet the same model achieves 10.1% err5. Some investigation showed that multiple pretrained models, including but not limited to VGG-16, VGG-19 and VGG-F have incorrect net.normalization.border. It is set to [0 0], while in fact it should be [32 32]. This results in input images being resized (prior to cropping) so that the smaller dimension is 224, instead of 256. In April 2015 this issue was obscured by commit https://github.com/vlfeat/matconvnet/commit/58ba8b008cd789d3369e02b10fbf75943ff89631 which added the following line to cnn_imagenet_evaluate:

net.normalization.border = [256 256] - net.normalization.imageSize(1:2) ;

Since that commit, cnn_imagenet_evaluate works correctly, while any other code using the same pretrained models achieves inferior results. I think this is a very confusing behavior. IMO, the net.normalization.border field of the pretrained models should be patched, and the offending line from cnn_imagenet_evaluate should be removed.

lenck commented 8 years ago

Hi, you are right, the pre-trained models seem to have this issue. And sorry that it took so long to reply.

However I think a cleaner solution would be to remove the border argument from the pre-trained model...

vedaldi commented 8 years ago

Hi, ok, thanks for noticing. This completely escaped me (good thing that the performance increases at least).

lenck commented 8 years ago

I think it actually does not affect the evaluation code, as we currently have 9.9% err5. But the border field in the models is probably unnecessary.

vedaldi commented 8 years ago

Well, it is necessary to know exactly how the model was trained, because from it you can deduce how large the input image was.

On 19 Apr 2016, at 19:44, Karel Lenc notifications@github.com wrote:

I think it actually does not affect the evaluation code, as we currently have 9.9% err5. But the border field in the models is probably unnecessary.

— You are receiving this because you commented. Reply to this email directly or view it on GitHub https://github.com/vlfeat/matconvnet/issues/296#issuecomment-212064482