open-mmlab / mmdetection

OpenMMLab Detection Toolbox and Benchmark
https://mmdetection.readthedocs.io
Apache License 2.0
29.43k stars 9.43k forks source link

loss_cls: 0.0000, loss_bbox: 0.0000 when training YOLOX on my own dataset #6911

Open pumpkin33498 opened 2 years ago

pumpkin33498 commented 2 years ago

我用YOLOX训练自己的数据集的时候一直显示loss_cls: 0.0000, loss_bbox: 0.0000,然后测试的时候会有ERROR The testing results of the whole dataset is empty 请问可能是什么原因?

ZouRuia commented 2 years ago

我也遇到了,但是没有提示是error,loss_cls,loss_bbox都有值

VJoer commented 2 years ago

请问这个问题你解决了吗?我也遇到了同样的问题

VJoer commented 2 years ago

我用YOLOX训练自己的数据集的时候一直显示loss_cls: 0.0000, loss_bbox: 0.0000,然后测试的时候会有ERROR The testing results of the whole dataset is empty 请问可能是什么原因? 找到原因了,在coco.py文件中要设置类别,因为我自己的CoCo数据集也是80类,所以没有改,但是类别的名称和CoCo2017类别名称不一样,你需要将类别名称改一下 image. 参考这个链接

ZouRuia commented 2 years ago

不知道为啥,在运行一遍,正常跑通了。没有改任何东西。

ZouRuia commented 2 years ago

之前的是这样的: [>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>] 5000/5000, 17.4 task/s, elapsed: 287s, ETA: 0s writing results to ./result.pkl

Evaluating bbox... Loading and preparing results... The testing results of the whole dataset is empty. 我用的都是coco2017的数据,没有变过,之前跑过faster_rcnn,mask_rcnn,yolo,都可以,跑fcos_50也可以,用同样的学习率,gpu个数去跑fcos_101就出现了上面的问题,但是没有改什么,再跑一遍,就又跑通了。

VJoer commented 2 years ago

之前的是这样的: [>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>] 5000/5000, 17.4 task/s, elapsed: 287s, ETA: 0s writing results to ./result.pkl

Evaluating bbox... Loading and preparing results... The testing results of the whole dataset is empty. 我用的都是coco2017的数据,没有变过,之前跑过faster_rcnn,mask_rcnn,yolo,都可以,跑fcos_50也可以,用同样的学习率,gpu个数去跑fcos_101就出现了上面的问题,但是没有改什么,再跑一遍,就又跑通了。

你是用自己训练的模型还是官方给的预训练模型跑测试的?

ZouRuia commented 2 years ago

没有用预训练模型,自己跑出来的模型测试的。

ZouRuia commented 2 years ago

我的loss_cls: 0.0000, loss_bbox: 0.0000这两个值不是0.0000,是有值的,还正常的值。

pumpkin33498 commented 2 years ago

我用YOLOX训练自己的数据集的时候一直显示loss_cls: 0.0000, loss_bbox: 0.0000,然后测试的时候会有ERROR The testing results of the whole dataset is empty 请问可能是什么原因? 找到原因了,在coco.py文件中要设置类别,因为我自己的CoCo数据集也是80类,所以没有改,但是类别的名称和CoCo2017类别名称不一样,你需要将类别名称改一下 image. 参考这个链接

谢谢,我看了一下我的类别应该改了QAQ

VJoer commented 2 years ago

没有用预训练模型,自己跑出来的模型测试的。

我的问题是和楼主一样的训练自己的数据集的时候一直显示loss_cls: 0.0000, loss_bbox: 0.0000,然后测试的时候出现The testing results of the whole dataset is empty.,然后我就在coco.py中增加了类名解决了问题,你可以尝试在coco.py中增加类名,然后在coco.py中添加test=dict( samples_per_gpu=32, classes=classes,

JarvisKevin commented 2 years ago

同!跑coco2017都不行

hhaAndroid commented 2 years ago

@VJoer You can see the training log we released, it is normal.

Snixells commented 2 years ago

Hey there! I had the same problem and maybe the solution helps you too: In your COCO Annotations you have to specify an area (how big the boxes are). If you set that to 0 the MMDet training ignores these boxes. That's why the loss quickly goes to 0 (because the boxes do not exist and it is trained to not find anything at all)

kaphleamrit2 commented 2 years ago

@Snixells How to specify an area in COCO annotation. It's already there in my datasets. Could you please explain it again? Thanks. Screenshot 2022-03-15 192149

fire717 commented 2 years ago

遇到了同样的问题,coco.py中的类别数量除了要保持一致,名称也要一样才行。

Kinglee90 commented 2 years ago

### 可以尝试把lr调低,有可能是lr过大,导致梯度消失

52THANOS commented 1 year ago

我跑yolov7也遇到这个问题了,但是我修改classes也没有解决

1127244933 commented 1 year ago

同样遇到这样的问题,第一种文档里说可能是正常现象,第二种属于类别没有改,我用的yolov5,具体是yolov5cocodataset继承了mmdet中cocodataset的类别,在其中修改类别就可以。

kennyadelaide commented 1 year ago

Hey there! I had the same problem and maybe the solution helps you too: In your COCO Annotations you have to specify an area (how big the boxes are). If you set that to 0 the MMDet training ignores these boxes. That's why the loss quickly goes to 0 (because the boxes do not exist and it is trained to not find anything at all)

sir! how to specify the bbox size ? e.g. the areas field is?

sjiang95 commented 2 months ago

Hey there! I had the same problem and maybe the solution helps you too: In your COCO Annotations you have to specify an area (how big the boxes are). If you set that to 0 the MMDet training ignores these boxes. That's why the loss quickly goes to 0 (because the boxes do not exist and it is trained to not find anything at all)

This solution solved my problem. Thanks!

My dataset has no instance mask. So I set the area to the area of the bbox.

See https://github.com/open-mmlab/mmdetection/blob/cfd5d3a985b0249de009b67d04f37263e11cdf3d/mmdet/datasets/coco.py#L146-L147 it seems that mmdetection ignores the corresponding bbox if area <= 0.

But, refer to the definition of the keyword "area" of COCO dataset https://github.com/cocodataset/cocoapi/issues/36#issuecomment-279472475, the area means segmentation area. It makes no sense to ignore bboxes if the custom dataset contains no segmentation mask.

Should this be considered as a bug or feature of mmdetection? @hhaAndroid

gs-max commented 1 month ago

我的yolox跑标准的coco数据集一点事没有,自己的数据集就会显示这两项损失为0.能使的方法都试了,哎。