Error testing dataset PASCAL VOC2007

[Curry1201]

When I tested PASACL VOC2007, I referenced it (https://github.com/open-mmlab/mmdetection/issues/132).

Step 1: In the case where the coco dataset can be tested normally, I use pascal_voc.py to convert VOC2007 into a format supported by mmdetection (voc07_train.pkl ,voc07_trainval.pkl and voc07_val.pkl). The directories are mmdetection/data/VOC2007/annotations and mmdetection/data/VOC2007/JPEGImages.

Step 2: I modified configs/mask_rcnn_r50_fpn_1x.py The specific modifications are as follows:

dataset settings

Dataset_type = 'CocoDataset' Data_root = 'data/VOC2007/' Img_norm_cfg = dict( Mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_rgb=True) Data = dict( Imgs_per_gpu=2, Workers_per_gpu=2, Train=dict( Type=dataset_type, Ann_file=data_root + 'annotations/voc07_train.pkl', Img_prefix=data_root + 'JPEGImages/', Img_scale=(1333, 800), Img_norm_cfg=img_norm_cfg, Size_divisor=32, Flip_ratio=0.5, With_mask=True, With_crowd=True, With_label=True), Val=dict( Type=dataset_type, Ann_file=data_root + 'annotations/voc07_val.pkl', Img_prefix=data_root + 'JPEGImages/', Img_scale=(1333, 800), Img_norm_cfg=img_norm_cfg, Size_divisor=32, Flip_ratio=0, With_mask=True, With_crowd=True, With_label=True), Test=dict( Type=dataset_type, Ann_file=data_root + 'annotations/voc07_trainval.pkl', Img_prefix=data_root + 'JPEGImages/', Img_scale=(1333, 800), Img_norm_cfg=img_norm_cfg, Size_divisor=32, Flip_ratio=0, With_mask=False, With_label=False, Test_mode=True))

Step 3: Run the test code " python3 tools/test.py configs/mask_rcnn_r50_fpn_1x.py checkpoints/mask_rcnn/mask_rcnn_r50_fpn_1x_20181010-069fa190.pth --gpus 2 --out results/test_voc_1.pkl ". An error occurred at this time: UnicodeDecodeError: 'utf-8' codec can't decode byte 0x80 in position 0: invalid start byte the information is as follows: Power@ubuntu:~/mmdetection$ python3 tools/test.py configs/mask_rcnn_r50_fpn_1x.py checkpoints/mask_rcnn/mask_rcnn_r50_fpn_1x_20181010-069fa190.pth --gpus 2 --out results/test_voc_1.pkl Loading annotations into memory... Traceback (most recent call last): File "tools/test.py", line 123, in Main() File "tools/test.py", line 71, in main Dataset = obj_from_dict(cfg.data.test, datasets, dict(test_mode=True)) File "/usr/local/lib/python3.6/dist-packages/mmcv-0.2.0-py3.6.egg/mmcv/runner/utils.py", line 72, in obj_from_dict Return obj_type(**args) File "/usr/local/lib/python3.6/dist-packages/mmdet-0.5.4+65a2e5e-py3.6.egg/mmdet/datasets/custom.py", line 49, in init Self.img_infos = self.load_annotations(ann_file) File "/usr/local/lib/python3.6/dist-packages/mmdet-0.5.4+65a2e5e-py3.6.egg/mmdet/datasets/coco.py", line 10, in load_annotations Self.coco = COCO(ann_file) File "/usr/local/lib/python3.6/dist-packages/pycocotools-2.0.0-py3.6-linux-x86_64.egg/pycocotools/coco.py", line 84, in init Dataset = json.load(open(annotation_file, 'r')) File "/usr/lib/python3.6/json/init.py", line 296, in load Return loads(fp.read(), File "/usr/lib/python3.6/codecs.py", line 321, in decode (result, consumed) = self._buffer_decode(data, self.errors, final) UnicodeDecodeError: 'utf-8' codec can't decode byte 0x80 in position 0: invalid start byte

Step 4: Finding the information on the Internet may be due to the fact that the encoding format of the pkl file is not UTF8, so I converted the encoding of the three pkl files into UTF8, and then re-run the code of the third step. However, a new error has occurred: json.decoder.JSONDecodeError: Unexpected UTF-8 BOM (decode using utf-8-sig): line 1 column 1 (char 0) the information is as follows: Power@ubuntu:~/mmdetection$ python3 tools/test.py configs/mask_rcnn_r50_fpn_1x.py checkpoints/mask_rcnn/mask_rcnn_r50_fpn_1x_20181010-069fa190.pth --gpus 2 --out results/test_voc_1.pkl Loading annotations into memory... Traceback (most recent call last): File "tools/test.py", line 123, in Main() File "tools/test.py", line 71, in main Dataset = obj_from_dict(cfg.data.test, datasets, dict(test_mode=True)) File "/usr/local/lib/python3.6/dist-packages/mmcv-0.2.0-py3.6.egg/mmcv/runner/utils.py", line 72, in obj_from_dict Return obj_type(args) File "/usr/local/lib/python3.6/dist-packages/mmdet-0.5.4+65a2e5e-py3.6.egg/mmdet/datasets/custom.py", line 49, in init Self.img_infos = self.load_annotations(ann_file) File "/usr/local/lib/python3.6/dist-packages/mmdet-0.5.4+65a2e5e-py3.6.egg/mmdet/datasets/coco.py", line 10, in load_annotations Self.coco = COCO(ann_file) File "/usr/local/lib/python3.6/dist-packages/pycocotools-2.0.0-py3.6-linux-x86_64.egg/pycocotools/coco.py", line 84, in init Dataset = json.load(open(annotation_file, 'r')) File "/usr/lib/python3.6/json/init.py", line 299, in load Parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, kw) File "/usr/lib/python3.6/json/init.py", line 344, in loads s, 0) json.decoder.JSONDecodeError: Unexpected UTF-8 BOM (decode using utf-8-sig): line 1 column 1 (char 0)

Step 5: I know that it may be because the converted file contains BOM characters, so I am based on the above error message, /usr/local/lib/python3.6/dist-packages/pycocotools-2.0.0-py3. 6-linux-x86_64.egg/pycocotools/coco.py has been modified. I added the following two lines of code under the code dataset = json.load(open(annotation_file, 'r')) . If annotation_file.startswith(u'\ufeff'): annotation_file = annotation_file.encode('utf8')[3:].decode('utf8') Then I re-run the test code, but still reported an error, and it was consistent with the error in the fourth step.

Can you help me analyze the following error causes?

[hellock]

If you used the converting script to convert VOC annotations to a pkl file, then you need to use CustomDataset instead of CocoDataset.

Just for reference, Faster R-CNN w/ FPN trained on voc07+12 trainval get 79.7 mAP on voc07 test.

[Curry1201]

Thanks.

[polar99]

@hellock What is the config file for Faster R-CNN w/ FPN trained on voc07+12 trainval get 79.7 mAP on voc07 test. Is ./configs/pascal_voc/faster_rcnn_r50_fpn_1x_voc0712.py?