Question about map

[TianhangTang]

Hello, I tried to use PASCAL VOC2007 trainval to train this orignal FPN_resnet_101 without any changes, and test on VOC2007 test, I found the test map is 78.57%. So I wonder if you did some improvement on FPN recently, cause the test result on VOC2007 in your cfgs.py is 76.14%.

[TianhangTang]

I set the parameter "USE_07_METRIC=True" in the cfgs.py, which means using 11 point method to calculate ap, and the test map is 76.003%, I think this might be the reason of higher map I got before, emmm, which one is the correct map, I am confused.

[TianhangTang]

So, is the VOC2007 and VOC2012 map given by cfgs.py calculated by setting parameter "USE_07_METRIC=True"?

[yangxue0827]

In general, if you use the voc2007 dataset, set USE_07_METRIC to True, and if you use the voc2012 dataset, set it to False. The former is generally lower than the latter, and the latter is more accurate. In addition, the COCO data evaluation script should be consistent with voc2012. @TianhangTang

[TianhangTang]

Thank you very much. Emmm, so the voc2012_test2007 map(78.57%) in cfgs.py is calculated by setting "USE_07_METRIC=False"?Because I made a little change based on FPN, and the voc2012 test on voc2007 map is 81.79%("USE_07_METRIC=False"), I want to see whether I made any improvement or not. If the voc2012 map(78.57%) is calculate by "USE_07_METRIC=True", then the real value should be larger, and I have to retrain the original FPN and do the comparative experiment, it is really time consuming.

[yangxue0827]

I wrote in cfgs.py whether to use USE_12_METRIC （line46） or USE_07_METRIC (line88) @TianhangTang

[yangxue0827]

USE_07_METRIC = False in the default cfgs.py, that is, the evaluation method of voc2012 is used @TianhangTang

[TianhangTang]

@yangxue0827 I got it, both map results are trained and evaluated on VOC2007, I mistook the former one as trained on VOC2012. Thanks again.