Created by Jiangtao Xie and Peihua Li
This repository contains the source code and models trained on ImageNet 2012 dataset for the following paper:
@article{Li2017,
author = {Peihua Li,Jiangtao Xie,Qilong Wang and Wangmeng Zuo},
title = {Is Second-order Information Helpful for Large-scale Visual Recognition?},
journal= {International Conference on Computer Vision (ICCV)},
year = {2017}
}
We proposed the second-order pooling to replace the common first-order, max/average pooling after the last conv. layer. The proposed networks, called MPN-COV ConvNets, achieved consistent, nontrivial improvements over their counterparts. The key to our method is Matrix Power Normalization of COVariance, which
appropriately exploits Riemannian geometry which allows zero eigenvalues, overcoming the downside of the well-known Log-Euclidean metric in this scenario.
You can visit our project page for more details.
Network | 224x224 1-crop |
224x224 10-crop |
GoogleDrive | BaiduCloud |
---|---|---|---|---|
MPN-COV-ResNet-50 | 22.27/6.35 | 21.16/5.58 | 186.8MB | 186.8MB |
MPN-COV-ResNet-101 | 21.17/5.70 | 19.71/5.01 | 270.7MB | 270.7MB |
MPN-COV-AlexNet | 38.37/17.14 | 34.97/14.60 | 567.0MB | 567.0MB |
MPN-COV-VGG-M | 34.63/14.64 | 31.81/12.52 | 581.6MB | 581.6MB |
MPN-COV-VGG-16 | 26.55/8.94 | 24.68/7.75 | 614.0MB | 614.0MB |
We developed our programs based on MatConvNet and Matlab 2015b, running under either Ubuntu 14.04.5 LTS and Windows 7. To implement MPN-COV layer, we adopt the eigenvalue decomposition algorithm on CPU in single-precision format, as its GPU version on the CUDA platform is much slower. Except for eigenvalue decomposition, all other operations in forward and backward propagations are performed using C++ on GPU. While writing code, we follow the convention of MatConvNet as closely as possible.
└── matconvnet_root_dir
└── matlab
├── src
│ ├── bits
│ │ ├── impl
│ │ │ ├── blashelper_cpu.hpp
│ │ │ ├── blashelper_gpu.hpp
│ │ │ ├── mpn_cov_cpu.cpp
│ │ │ ├── mpn_cov_gpu.cu
│ │ │ └── nnmpn_cov_blas.hpp
│ │ ├── nnmpn_cov.cpp
│ │ ├── nnmpn_cov.cu
│ │ └── nnmpn_cov.hpp
│ ├── vl_nnmpn_cov.cpp
│ └── vl_nnmpn_cov.cu
├── +dagnn
│ └── MPN_COV_Pool_C.m
└── EIG.m
We use mexCallMATLAB to call Matlab built-in eig function in our C/C++ mexFunction file. Actually, we write a matlab function called EIG.m
performing eigenvalue decomposition of a batch of covariance matrices.
MPN_COV_Pool_C.m
is object oriented, enabling adding the layer of MPN-COV when the wrapper of DagNN is used.
└── matconvnet_root_dir
└── matlab
├── vl_compilenn.m
└── simplenn
└── vl_simplenn.m
vl_compilenn.m
so that our MPN-COV code can be complied, and modified vl_simplenn.m
to support the layer interface when the wrapper of SimpleNN is used. >> git clone https://github.com/jiangtaoxie/MPN-COV
>> vl_compilenn('enableGpu', true, ...
'cudaRoot', '/Developer/NVIDIA/CUDA-8.0', ...
'cudaMethod', 'nvcc', ...
'enableCudnn', true, ...
'cudnnRoot', 'local/cudnn-rc4') ;
net.layers{end+1} = struct('type','mpn_cov',...
'name','mpn_cov_pool',...
'method',[],...
'regu_method','power',...
'alpha', 0.5,...
'epsilon', 0);
name = 'mpn_cov_pool';
net.addLayer(name , ...
dagnn.MPN_COV_Pool_C('method', [],...
'regu_method', 'power', ...
'alpha', 0.5,...
'epsilon', 0), ...
lastAdded.var, ...
{name, [name, '_aux_S'], [name, '_aux_V'],[name,'_aux_D']});
lastAdded.var = name;
In our demo code, we implement MPN-COV AlexNet, VGG-M and VGG-VD under SimpleNN framework, and MPN-COV ResNet under DagNN framework.
'method'
: It is reserved for future use.'regu_method'
: We introduced three normalization methods in the paper, namely, MPN-COV
,MPN-COV+matrix-l2
,MPN-COV+matrix-Fro
. As the latter two normalizations produced unsatisfactory performance, we only support MPN-COV, designated by 'power'
.'alpha'
: It denotes the exponent of matrix power function(equivalently, the power of eigenvalues, see the paper), whose values should be positive. The default value is 0.5 producing the best performance.'epsilon'
: It is a small positive number added to eigenvalues of covariance matrices. It is set to 0 as the Power-E metric allows the eigenvalue to be non-negative.If you have any questions or suggestions, please contact us
jiangtaoxie@mail.dlut.edu.cn