glenn-jocher
commented
5 years ago @AbdallahGaber00 The mAP is working as intended and as reported. pycocotools mAP reported at the bottom is currently 0.607.
0.595 is the internal mAP reported by this repo, which is a little less than the official pycocotools mAP.
python3 test.py --save-json --img-size 608
Namespace(batch_size=16, cfg='cfg/yolov3-spp.cfg', conf_thres=0.001, data_cfg='data/coco.data', img_size=608, iou_t
hres=0.5, nms_thres=0.5, save_json=True, weights='weights/yolov3-spp.weights')
Using CUDA device0 _CudaDeviceProperties(name='Tesla V100-SXM2-16GB', total_memory=16130MB)
Reading labels: 100%|████████████████████████████████████████████████████████| 5000/5000 [00:03<00:00, 1337.86it/s]
Class Images Targets P R mAP F1
Computing mAP: 100%|█████████████████████████████████████████████████████████████| 313/313 [06:24<00:00, 1.53it/s]
all 5e+03 3.58e+04 0.119 0.79 0.595 0.201
person 5e+03 1.09e+04 0.162 0.885 0.758 0.274
bicycle 5e+03 316 0.0698 0.791 0.561 0.128
car 5e+03 1.67e+03 0.0771 0.854 0.652 0.141
motorcycle 5e+03 391 0.163 0.834 0.714 0.273
airplane 5e+03 131 0.184 0.901 0.858 0.305
bus 5e+03 261 0.205 0.854 0.798 0.331
train 5e+03 212 0.178 0.892 0.817 0.297
truck 5e+03 352 0.106 0.668 0.502 0.184
boat 5e+03 475 0.0958 0.779 0.523 0.171
traffic light 5e+03 516 0.0529 0.791 0.552 0.0992
fire hydrant 5e+03 83 0.191 0.928 0.885 0.316
stop sign 5e+03 84 0.0947 0.893 0.829 0.171
parking meter 5e+03 59 0.0729 0.712 0.621 0.132
bench 5e+03 473 0.0347 0.66 0.364 0.066
bird 5e+03 469 0.0884 0.684 0.525 0.157
cat 5e+03 195 0.306 0.872 0.781 0.453
dog 5e+03 223 0.256 0.87 0.815 0.396
horse 5e+03 305 0.173 0.898 0.833 0.29
sheep 5e+03 321 0.25 0.835 0.717 0.384
cow 5e+03 384 0.188 0.831 0.724 0.307
elephant 5e+03 284 0.25 0.954 0.907 0.396
bear 5e+03 53 0.42 0.887 0.848 0.57
zebra 5e+03 277 0.247 0.935 0.871 0.391
giraffe 5e+03 170 0.24 0.912 0.884 0.379
backpack 5e+03 384 0.0489 0.721 0.387 0.0916
umbrella 5e+03 392 0.102 0.86 0.643 0.182
handbag 5e+03 483 0.0268 0.669 0.271 0.0516
tie 5e+03 297 0.0673 0.825 0.598 0.124
suitcase 5e+03 310 0.135 0.819 0.61 0.232
frisbee 5e+03 109 0.197 0.899 0.856 0.323
skis 5e+03 282 0.067 0.745 0.418 0.123
snowboard 5e+03 92 0.0948 0.739 0.527 0.168
sports ball 5e+03 236 0.0885 0.784 0.685 0.159
kite 5e+03 399 0.181 0.812 0.596 0.296
baseball bat 5e+03 125 0.0836 0.728 0.541 0.15
baseball glove 5e+03 139 0.0762 0.799 0.639 0.139
skateboard 5e+03 218 0.113 0.83 0.77 0.2
surfboard 5e+03 266 0.0928 0.808 0.658 0.167
tennis racket 5e+03 183 0.145 0.863 0.755 0.248
bottle 5e+03 966 0.0744 0.778 0.515 0.136
wine glass 5e+03 366 0.108 0.754 0.565 0.19
cup 5e+03 897 0.0877 0.794 0.559 0.158
fork 5e+03 234 0.0612 0.688 0.461 0.112
knife 5e+03 291 0.0459 0.646 0.331 0.0857
spoon 5e+03 253 0.0374 0.66 0.294 0.0708
bowl 5e+03 620 0.091 0.81 0.505 0.164
banana 5e+03 371 0.0893 0.709 0.341 0.159
apple 5e+03 158 0.0506 0.696 0.223 0.0943
sandwich 5e+03 160 0.112 0.719 0.495 0.193
orange 5e+03 189 0.0539 0.593 0.26 0.0988
broccoli 5e+03 332 0.0994 0.762 0.374 0.176
carrot 5e+03 346 0.063 0.671 0.296 0.115
hot dog 5e+03 164 0.144 0.61 0.477 0.233
pizza 5e+03 224 0.117 0.821 0.67 0.204
donut 5e+03 237 0.141 0.764 0.597 0.238
cake 5e+03 241 0.0993 0.672 0.522 0.173
chair 5e+03 1.62e+03 0.0674 0.735 0.461 0.124
couch 5e+03 236 0.126 0.767 0.588 0.216
potted plant 5e+03 431 0.0561 0.805 0.478 0.105
bed 5e+03 195 0.169 0.856 0.731 0.282
dining table 5e+03 634 0.0627 0.792 0.498 0.116
toilet 5e+03 179 0.242 0.939 0.829 0.385
tv 5e+03 257 0.131 0.926 0.817 0.229
laptop 5e+03 237 0.182 0.857 0.752 0.301
mouse 5e+03 95 0.0848 0.884 0.742 0.155
remote 5e+03 241 0.0694 0.817 0.574 0.128
keyboard 5e+03 117 0.0868 0.889 0.73 0.158
cell phone 5e+03 291 0.0417 0.715 0.46 0.0788
microwave 5e+03 88 0.213 0.875 0.782 0.342
oven 5e+03 142 0.0788 0.803 0.541 0.143
toaster 5e+03 11 0.1 0.818 0.417 0.178
sink 5e+03 211 0.0697 0.834 0.596 0.129
refrigerator 5e+03 107 0.0922 0.935 0.78 0.168
book 5e+03 1.08e+03 0.0587 0.647 0.201 0.108
clock 5e+03 292 0.0837 0.87 0.75 0.153
vase 5e+03 353 0.0954 0.793 0.562 0.17
scissors 5e+03 56 0.0539 0.696 0.431 0.1
teddy bear 5e+03 245 0.158 0.853 0.669 0.267
hair drier 5e+03 11 0.0488 0.182 0.121 0.0769
toothbrush 5e+03 77 0.0466 0.714 0.321 0.0876
loading annotations into memory...
Done (t=5.03s)
creating index...
index created!
Loading and preparing results...
DONE (t=3.33s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=46.97s).
Accumulating evaluation results...
DONE (t=6.16s).
Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.367
Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.607
Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.386
Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.209
Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.394
Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.488
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.296
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.465
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.494
Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.332
Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.519
Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.619
I've downloaded the weights for yolov3-spp.weights and run test.pt as the following
!python3 test.py --save-json --img-size 608still, I get mAP for all classes = 0.595 not 60.7 !! what's wrong ?