Discusses appropriate loss functions for deep learning models, such as object detection, face recognition, and image segmentation. The downsides of traditional loss functions and recent advancements in designing novel loss functions are discussed to address problems such as imbalanced data, uncertain distribution, overlapped bounding boxes, and over-fitting. The paper discusses potential future research directions in this field.
Key Points
The loss function usually measures the accuracy, similarity, or goodness of fit between the predicted value and ground-truth.
A carefully prepared loss function is one that addresses the challenges of the specific deep learning model. It can improve the training performance of the neural network significantly.
object detectors usually have two sub-tasks: object classification and object localization. A well-selected loss function will contribute to the success of these tasks.
Cross entropy is used for image classification, object detection, and segmentation. In these tasks, cross entropy is used to measure the difference between the predicted probabilities of the classes and the true probabilities of the classes, allowing for optimization of the model parameters. Cross entropy is also used to measure the performance of a neural network on a given image dataset.
A softmax layer is a type of activation layer in a neural network used for classification. It is used to output a probability distribution over the classes. The softmax layer calculates the probability of each class by applying the softmax function to the output of the previous layer. The softmax function takes a vector of arbitrary real values and normalizes it into a probability distribution by taking the exponent of each element and then normalizing the resulting vector.
Citation
Yingjie Tian, Duo Su, Stanislao Lauria, Xiaohui Liu,
Recent advances on loss functions in deep learning for computer vision,
Neurocomputing,
Volume 497,
2022,
Pages 129-158,
ISSN 0925-2312,
https://doi.org/10.1016/j.neucom.2022.04.127.
Title
Recent advances on loss functions in deep learning for computer vision
URL
https://www.sciencedirect.com/science/article/pii/S0925231222005239
Summary
Discusses appropriate loss functions for deep learning models, such as object detection, face recognition, and image segmentation. The downsides of traditional loss functions and recent advancements in designing novel loss functions are discussed to address problems such as imbalanced data, uncertain distribution, overlapped bounding boxes, and over-fitting. The paper discusses potential future research directions in this field.
Key Points
Citation
Yingjie Tian, Duo Su, Stanislao Lauria, Xiaohui Liu, Recent advances on loss functions in deep learning for computer vision, Neurocomputing, Volume 497, 2022, Pages 129-158, ISSN 0925-2312, https://doi.org/10.1016/j.neucom.2022.04.127.
Repo link