Higher accuracy when running test than in README

[xsacha]

Hi, is the README accuracy outdated? I ran the test model (TP_R_256x192_d256_h1024_enc4_mh8) as-is with latest pytorch:

Test: [0/794] Time 1.056 (1.056) Loss 0.0001 (0.0001) Accuracy 0.977 (0.977) Test: [100/794] Time 0.077 (0.092) Loss 0.0003 (0.0003) Accuracy 0.898 (0.866) Test: [200/794] Time 0.082 (0.086) Loss 0.0004 (0.0003) Accuracy 0.841 (0.869) Test: [300/794] Time 0.078 (0.084) Loss 0.0002 (0.0003) Accuracy 0.965 (0.866) Test: [400/794] Time 0.078 (0.083) Loss 0.0002 (0.0003) Accuracy 0.866 (0.866) Test: [500/794] Time 0.082 (0.082) Loss 0.0002 (0.0003) Accuracy 0.947 (0.868) Test: [600/794] Time 0.080 (0.082) Loss 0.0006 (0.0003) Accuracy 0.841 (0.867) Test: [700/794] Time 0.078 (0.081) Loss 0.0004 (0.0003) Accuracy 0.918 (0.867) => writing results json to output/coco/transpose_r/TP_R_256x192_d256_h1024_enc4_mh8/results/keypoints_val2017_results_0.json Loading and preparing results... DONE (t=0.33s) creating index... index created! Running per image evaluation... Evaluate annotation type keypoints DONE (t=2.71s). Accumulating evaluation results... DONE (t=0.12s). Average Precision (AP) @[ IoU=0.50:0.95	area= all	maxDets= 20 ] = 0.751 Average Precision (AP) @[ IoU=0.50	area= all	maxDets= 20 ] = 0.926 Average Precision (AP) @[ IoU=0.75	area= all	maxDets= 20 ] = 0.826 Average Precision (AP) @[ IoU=0.50:0.95	area=medium	maxDets= 20 ] = 0.719 Average Precision (AP) @[ IoU=0.50:0.95	area= large	maxDets= 20 ] = 0.796 Average Recall (AR) @[ IoU=0.50:0.95	area= all	maxDets= 20 ] = 0.778 Average Recall (AR) @[ IoU=0.50	area= all	maxDets= 20 ] = 0.932 Average Recall (AR) @[ IoU=0.75	area= all	maxDets= 20 ] = 0.841 Average Recall (AR) @[ IoU=0.50:0.95	area=medium	maxDets= 20 ] = 0.743 Average Recall (AR) @[ IoU=0.50:0.95	area= large	maxDets= 20 ] = 0.830	Arch	AP	Ap .5	AP .75	AP (M)	AP (L)	AR	AR .5	AR .75	AR (M)	AR (L)
transpose_r	0.751	0.926	0.826	0.719	0.796	0.778	0.932	0.841	0.743	0.830	**

From README:

Model	Input size	FPS*	GFLOPs	AP	Ap .5	AP .75	AP (M)	AP (L)	AR	AR .5	AR .75	AR (M)	AR (L)
TransPose-R-A3	256x192	141	8.0	0.717	0.889	0.788	0.680	0.786	0.771	0.930	0.836	0.727	0.835
TransPose-R-A4	256x192	138	8.9	0.726	0.891	0.799	0.688	0.798	0.780	0.931	0.845	0.735	0.844

[yangsenius]

Hi, @xsacha. 0.751 AP is achieved by using GT human boxes. The 0.726AP is achieved when python tools/test.py --cfg experiments/coco/transpose_r/TP_R_256x192_d256_h1024_enc4_mh8.yaml TEST.USE_GT_BBOX False

[xsacha]

Oh. Thanks for that.