centernet face 训练时 loss = inf

yuqj1991 / caffe_train

caffe train face licenseplate reID action ocr centernet

23 stars 12 forks source link

centernet face 训练时 loss = inf #6

Closed ChHanXiao closed 4 years ago

ChHanXiao commented 4 years ago

练不起来loss 一直是inf，有什么调参技巧吗 I0419 17:36:08.272820 7887 CenternetLossLayer.cpp:213] total loss: nan, loc loss: inf, conf loss: nan, normalizer: 49, num_classes: 1, output_width: 160, output_height: 160 cuda diff_sumarize_a: nan, count: 0.000000
cuda diff_sumarize_a: nan, count: 0.000000
cuda diff_sumarize_a: nan, count: 0.000000
cuda diff_sumarize_a: nan, count: 0.000000
cuda diff_sumarize_a: nan, count: 0.000000

yuqj1991 commented 4 years ago

是训练的人脸吗？人脸的话，我用的是widerface数据集，不应该的啊

ChHanXiao commented 4 years ago

也是用的widerface，SSD可以训练，就这个不行

yuqj1991 commented 4 years ago

我测试了，是我的问题，我已将gpu版本的计算loss屏蔽了，我一开始是使用cpu版loss的，后来加上gpu版loss没有测试

yuqj1991 commented 4 years ago

你可重新更新再试试