low mAP value on PASCAL VOC 2007 dataset

[guoqiang0148666]

Thank you for your excellent works!

l am freshman to simpledet. l follow bellow commands to train and test faster rcnn on pascal voc 2007 dataset. l think these procedures are ok, but l got very low mAP , l don't know what's went wrong, l hope someone can help me, any suggestion will be welcomed!

Thanks!

1. create pascal voc roidb

   python utils/create_voc_roidb.py --data-dir /media/guo/DataUbuntu/VOCdevkit0712/VOC2007/ --split trainval
   python utils/create_voc_roidb.py --data-dir /media/guo/DataUbuntu/VOCdevkit0712/VOC2007/ --split test

   They create _trainval.roidb and _test.roidb in data/cache dir, l rename them to pascal_voc_train2007.roidb and pascal_voc_val2007.roidb

2. create label_map file

   {"__backgroud__":0,"aeroplane":1,"bicycle":2,"bird":3,"boat":4,"bottle":5,"bus":6,"car":7,"cat":8,"chair":9,"cow":10,"diningtable":11,
   "dog":12,"horse":13,"motorbike":14,"person":15,"pottedplant":16,"sheep":17,"sofa":18,"train":19,"tvmonitor":20}

3. train and test faster rcnn

   # train
   python detection_train.py --config config/faster_r50v1c4_c5_512roi_1x_voc07.py
   # test 
   python detection_test.py --config config/faster_r50v1c4_c5_512roi_1x_voc07.py

   my config file is modifed from 'config/faster_r50v1c4_c5_512roi_1x.py', the different is as follow :

   
   diff --git a/config/faster_r50v1c4_c5_512roi_1x.py b/config/faster_r50v1c4_c5_512roi_1x.py
   index 069a7e5..61ab6f5 100644
   --- a/config/faster_r50v1c4_c5_512roi_1x.py
   +++ b/config/faster_r50v1c4_c5_512roi_1x.py
   @@ -17,7 +17,8 @@ def get_config(is_train):
    class KvstoreParam:
        kvstore     = "local"
        batch_image = General.batch_image
   -        gpus        = [0, 1, 2, 3, 4, 5, 6, 7]
   +        #gpus        = [0, 1, 2, 3, 4, 5, 6, 7]
   +        gpus        = [0]
        fp16        = General.fp16

@@ -77,7 +78,7 @@ def get_config(is_train): class BboxParam: fp16 = General.fp16 normalizer = NormalizeParam.normalizer

• num_class = 1 + 80
• num_class = 1 + 20 image_roi = 512 batch_image = General.batch_image

@@ -96,9 +97,9 @@ def get_config(is_train):

 class DatasetParam:
     if is_train:

• image_set = ("coco_train2017", )
• image_set = ("pascal_voc_train2007", ) else:
• image_set = ("coco_val2017", )

• image_set = ("pascal_voc_val2007", )

  backbone = Backbone(BackboneParam) neck = Neck(NeckParam) @@ -168,7 +169,7 @@ def get_config(is_train): thr = 0.5

   class coco:

• annotation = "data/coco/annotations/instances_minival2014.json"

• annotation = None

  data processing

  class NormParam:

  4 **mAP results**

  Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.299 Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.625 Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.244 Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.088 Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.225 Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.357 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.287 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.423 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.434 Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.164 Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.352 Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.497 coco eval uses: 13.6

[huangzehao]

General VOC setting is train on voc0712 and test on voc07:

       if is_train:
            image_set = ("voc2007_trainval", "voc2012_trainval")
        else:
            image_set = ("voc2007_test", )

And we focus on AP@0.5 in VOC experiments.

[guoqiang0148666]

Thanks