By Weiyang Liu, Yandong Wen, Zhiding Yu, Meng Yang
We introduce a large-margin softmax (L-Softmax) loss for convolutional neural networks. L-Softmax loss can greatly improve the generalization ability of CNNs, so it is very suitable for general classification, feature embedding and biometrics (e.g. face) verification. We give the 2D feature visualization on MNIST to illustrate our L-Softmax loss.
The paper is published in ICML 2016 and also available at arXiv.
If the code helps your research, please consider to cite our work:
Large-Margin Softmax Loss for Convolutional Neural Networks
Weiyang Liu, Yandong Wen, Zhiding Yu and Meng Yang
Proceedings of The 33rd International Conference on Machine Learning. 2016: 507-516.
@inproceedings{liu2016large,
title={Large-Margin Softmax Loss for Convolutional Neural Networks},
author={Liu, Weiyang and Wen, Yandong and Yu, Zhiding and Yang, Meng},
booktitle={Proceedings of The 33rd International Conference on Machine Learning},
pages={507--516},
year={2016}
}
We greatly appreciate the contributions for all the third-party re-implementations!
The prototxt of LargeMarginInnerProduct layer is as follows:
layer {
name: "ip2"
type: "LargeMarginInnerProduct"
bottom: "ip1"
bottom: "label"
top: "ip2"
top: "lambda"
param {
name: "ip2"
lr_mult: 1
}
largemargin_inner_product_param {
num_output: 10 //number of outputs
type: QUADRUPLE //value of m
//only SINGLE (m=1), DOUBLE (m=2), TRIPLE (m=3) and QUADRUPLE (m=4) are available.
base: 1000
gamma: 0.000025
power: 35
iteration: 0
lambda_min: 0
//base, gamma, power and lambda_min are parameters of exponential lambda descent
weight_filler {
type: "msra"
}
}
include {
phase: TRAIN
}
}
For specific examples, please refer to myexamples/mnist folder.
If you have any questions, feel free to contact:
Copyright(c) Weiyang Liu and Yandong Wen All rights reserved.
MIT License
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED AS IS, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.