doubt about the pre-model precision and recall

[yayong-guan]

hi，i use the model you released COCO.res50.384x288.CPN snapshot_350.ckpt，set test_subset= True, the num is first 1000, the result is so low Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets= 20 ] = 0.111 Average Precision (AP) @[ IoU=0.50 | area= all | maxDets= 20 ] = 0.131 Average Precision (AP) @[ IoU=0.75 | area= all | maxDets= 20 ] = 0.119 Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets= 20 ] = 0.108 Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets= 20 ] = 0.117 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 20 ] = 0.113 Average Recall (AR) @[ IoU=0.50 | area= all | maxDets= 20 ] = 0.132 Average Recall (AR) @[ IoU=0.75 | area= all | maxDets= 20 ] = 0.120 Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets= 20 ] = 0.108 Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets= 20 ] = 0.121

this is because of released model is underfitting？i try to draw the keypoints model predicted，show result is not correct.

[chenyilun95]

Did you ever try res50.256x192 model ? If that's OK, then maybe I should re-submit the model.

[yayong-guan]

not yet，I will try other models.

[yayong-guan]

@chenyilun95 , i try res50.256x192 model and res101.384x288 model , result is similar Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets= 20 ] = 0.105 Average Precision (AP) @[ IoU=0.50 | area= all | maxDets= 20 ] = 0.119 Average Precision (AP) @[ IoU=0.75 | area= all | maxDets= 20 ] = 0.113 Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets= 20 ] = 0.131 Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets= 20 ] = 0.064 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 20 ] = 0.101 Average Recall (AR) @[ IoU=0.50 | area= all | maxDets= 20 ] = 0.113 Average Recall (AR) @[ IoU=0.75 | area= all | maxDets= 20 ] = 0.113 Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets= 20 ] = 0.132 Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets= 20 ] = 0.058

i just run the mptest.py in corresponding folder, not modify the code, whether need to modify part of code to get right result.

[chenyilun95]

The submitted models should give the same result in the table. I think there must be somthing wrong. Some other guys has tested the res50.256192 model and got the right result. I’m outside now and I’ll check it again maybe in late afternoon.

[yayong-guan]

It is not urgent，thanks，i will try to check.

[yayong-guan]

@chenyilun95 ，i got the right result， code is correct，close this.