Closed RangeKing closed 2 years ago
Good Work! Would you like to open an PR to merge your implementation? BTW, since this is an useful component, please add an unittest with your PR. Thanks!
Good Work! Would you like to open an PR to merge your implementation? BTW, since this is an useful component, please add an unittest with your PR. Thanks!
Thanks for the reply! I'll give it a try.
Describe your feature request LabelSmoothingCrossEntropyLoss增加可选参数 losses.py中(以DeiT为例)LabelSmoothingCrossEntropyLoss可选参数较少 https://github.com/BR-IDL/PaddleViT/blob/dd437d454501943929f29d3e85760fb8e577b284/image_classification/DeiT/losses.py#L21-L46
Describe the reference code or paper
Describe the possible solution 直接调用paddle.nn.functional.cross_entropy计算,可以设置更多参数。 调用前先利用paddle.nn.functional.one_hot将标签转为on-hot形式,再用 paddle.nn.functional.label_smooth将标签平滑,最后将paddle.nn.functional.cross_entropy的soft_label设为True即可实现。 代码如下:
目前,经过简单测试结果和现有方法计算结果一致。
Additional context Paddle ViT课程训练ResNet18作业中发现,过拟合比较严重,所以想尝试利用Label Smoothing方法缓解。但是搜索paddle api官方文档后发现没有专门的LabelSmoothingCrossEntropyLoss,利用paddle现成的one_hot和label_smooth函数实现了一下。