Closed sjiang95 closed 1 year ago
Reason found.
In PR #9078, a new entry seg_suffix
is added for class CustomDataset
. mmdetection/mmdet/datasets/custom.py at e71b499608e9c3ccd4211e7c815fa20eeedf18a2 · open-mmlab/mmdetection
def __init__(self,
ann_file,
pipeline,
classes=None,
data_root=None,
img_prefix='',
seg_prefix=None,
seg_suffix='.png', # new entry
proposal_file=None,
test_mode=False,
filter_empty_gt=True,
file_client_args=dict(backend='disk'))
But in mmdet/datasets/cityscapes.py#L264, the initialization of class CocoDataset
remains.
self_coco = CocoDataset(self.ann_file, self.pipeline.transforms,
None, self.data_root, self.img_prefix,
self.seg_prefix, self.proposal_file,
self.test_mode, self.filter_empty_gt)
There is an arguments mismatch between the arguments of self_coco = CocoDataset
in mmdet/datasets/cityscapes.py
with the new __init__
in mmdet/datasets/custom.py
. self.proposal_file
in CocoDataset
class in mmdet/datasets/cityscapes.py
is assigned to seg_suffix
in CustomDataset.__init__()
in mmdet/datasets/custom.py
.
I'll create a PR to fix this.
This bug is fixed by PR #9330.
Test result
2022-11-16 09:39:56,842 - mmdet - INFO - workflow: [('train', 1)], max: 1000 iters [15/10370]2022-11-16 09:39:56,843 - mmdet - INFO - Checkpoints will be saved to /home/quan/Documents/my_repo/semclDetection/work_dirs/faster_rcnn_r50_fpn_1x_cityscapes by HardDiskBackend.
2022-11-16 09:41:16,037 - mmdet - INFO - Iter [100/1000] lr: 1.988e-04, eta: 0:11:52, time: 0.792, data_time: 0.004, memory: 1039, loss_rpn_cls: 0.0519, loss_rpn_bbox: 0.0627, loss_cls: 1.1775, acc: 68.8086, loss_bbox: 0.4482, loss: 1.7402
2022-11-16 09:42:35,523 - mmdet - INFO - Iter [200/1000] lr: 3.986e-04, eta: 0:10:34, time: 0.795, data_time: 0.004, memory: 1104, loss_rpn_cls: 0.0441, loss_rpn_bbox: 0.0640, loss_cls: 0.4772, acc: 85.6465, loss_bbox: 0.3460, loss: 0.9313
2022-11-16 09:43:55,972 - mmdet - INFO - Iter [300/1000] lr: 5.984e-04, eta: 0:09:17, time: 0.804, data_time: 0.004, memory: 1117, loss_rpn_cls: 0.0448, loss_rpn_bbox: 0.0676, loss_cls: 0.3586, acc: 88.1172, loss_bbox: 0.2444, loss: 0.7155
2022-11-16 09:45:16,004 - mmdet - INFO - Iter [400/1000] lr: 7.982e-04, eta: 0:07:58, time: 0.800, data_time: 0.004, memory: 1117, loss_rpn_cls: 0.0413, loss_rpn_bbox: 0.0722, loss_cls: 0.3453, acc: 88.3613, loss_bbox: 0.2199, loss: 0.6787
2022-11-16 09:46:35,151 - mmdet - INFO - Saving checkpoint at 500 iterations
2022-11-16 09:46:35,819 - mmdet - INFO - Iter [500/1000] lr: 9.980e-04, eta: 0:06:38, time: 0.798, data_time: 0.004, memory: 1298, loss_rpn_cls: 0.0478, loss_rpn_bbox: 0.0702, loss_cls: 0.3276, acc: 88.7852, loss_bbox: 0.2130, loss: 0.6586
[>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>] 500/500, 3.0 task/s, elapsed: 168s, ETA: 0sloading annotations into memory...
Done (t=0.04s)
creating index...
index created!
loading annotations into memory...
Done (t=0.16s)
creating index...
index created! 2022-11-16 09:49:24,107 - mmdet - INFO - Evaluating bbox...
Loading and preparing results...
DONE (t=0.14s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=6.84s).
Accumulating evaluation results...
DONE (t=0.62s).
2022-11-16 09:49:31,801 - mmdet - INFO -
Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.139
Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=1000 ] = 0.297
Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=1000 ] = -1.000
Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.052
Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.187
Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.281
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.227
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=300 ] = 0.227
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=1000 ] = 0.227
Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.054
Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.261
Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.439
2022-11-16 09:49:31,831 - mmdet - INFO - Iter(val) [500] bbox_mAP: 0.1390, bbox_mAP_50: 0.2970, bbox_mAP_75: -1.0000, bbox_mAP_s: 0.0520, bbox_mAP_m: 0.1870, bbox_mAP_l: 0.2810, bbox_mAP_copypaste: 0.139 0.297 -1.000 0.052 0.187 0.281
2022-11-16 09:50:51,997 - mmdet - INFO - Iter [600/1000] lr: 1.000e-03, eta: 0:07:16, time: 2.562, data_time: 1.764, memory: 1298, loss_rpn_cls: 0.0417, loss_rpn_bbox: 0.0703, loss_cls: 0.3083, acc: 89.3789, loss_bbox: 0.1991, loss: 0.6194
2022-11-16 09:52:12,053 - mmdet - INFO - Iter [700/1000] lr: 1.000e-03, eta: 0:05:15, time: 0.801, data_time: 0.004, memory: 1298, loss_rpn_cls: 0.0372, loss_rpn_bbox: 0.0690, loss_cls: 0.3128, acc: 89.4688, loss_bbox: 0.1874, loss: 0.6064
2022-11-16 09:53:30,970 - mmdet - INFO - Iter [800/1000] lr: 1.000e-03, eta: 0:03:23, time: 0.789, data_time: 0.004, memory: 1298, loss_rpn_cls: 0.0370, loss_rpn_bbox: 0.0709, loss_cls: 0.3139, acc: 88.6914, loss_bbox: 0.1968, loss: 0.6186
2022-11-16 09:54:49,779 - mmdet - INFO - Iter [900/1000] lr: 1.000e-03, eta: 0:01:39, time: 0.788, data_time: 0.004, memory: 1298, loss_rpn_cls: 0.0385, loss_rpn_bbox: 0.0743, loss_cls: 0.3071, acc: 89.1699, loss_bbox: 0.1938, loss: 0.6136
2022-11-16 09:56:09,959 - mmdet - INFO - Saving checkpoint at 1000 iterations
2022-11-16 09:56:10,754 - mmdet - INFO - Exp name: faster_rcnn_r50_fpn_1x_cityscapes.py
2022-11-16 09:56:10,754 - mmdet - INFO - Iter [1000/1000] lr: 1.000e-03, eta: 0:00:00, time: 0.810, data_time: 0.004, memory: 1298, loss_rpn_cls: 0.0314, loss_rpn_bbox: 0.0671, loss_cls: 0.2992, acc: 89.1348, loss_bbox: 0.1924, loss: 0.5901
[>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>] 500/500, 3.0 task/s, elapsed: 168s, ETA: 0sloading annotations into memory...
Done (t=0.15s)
creating index...
index created!
loading annotations into memory...
Done (t=0.05s)
creating index...
index created!
2022-11-16 09:58:58,904 - mmdet - INFO - Evaluating bbox...
Loading and preparing results...
DONE (t=0.02s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=5.92s).
Accumulating evaluation results...
DONE (t=0.55s).
2022-11-16 09:59:05,573 - mmdet - INFO -
Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.206
Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=1000 ] = 0.398
Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=1000 ] = -1.000
Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.064
Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.240
Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.411
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.304
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=300 ] = 0.304
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=1000 ] = 0.304
Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.067
Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.340
Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.547
2022-11-16 09:59:05,606 - mmdet - INFO - Exp name: faster_rcnn_r50_fpn_1x_cityscapes.py
2022-11-16 09:59:05,606 - mmdet - INFO - Iter(val) [500] bbox_mAP: 0.2060, bbox_mAP_50: 0.3980, bbox_mAP_75: -1.0000, bbox_mAP_s: 0.0640, bbox_mAP_m: 0.2400, bbox_mAP_l: 0.4110, bbox_mAP_copypaste: 0.206 0.398 -1.000 0.064 0.240 0.411
Prerequisite
Task
I'm using the official example scripts/configs for the officially supported tasks/models/datasets.
Branch
master branch https://github.com/open-mmlab/mmdetection
Environment
env0
limited by insufficient GPU memory, img resolution is tuned to
env1
On both env0&1, pytorch is installed by
Environment variables on env1:
$PATH
$LD_LIBRARY_PATH
$PYTHONPATH
Reproduces the problem - code sample
The cityscapes dataset is converted by 1: Inference and train with existing models and standard datasets — MMDetection 2.25.1 documentation
Single gpu (on env0):
Distributed (on env1):
Reproduces the problem - command or script
same as above
Reproduces the problem - error message
The error messages are same on env0&1.
Additional information
Training and evaluation on voc and coco are totally fine. For cityscapes, everything works like a charm in training stage, so I believe the dataset conversion has nothing to do with the error.