YOLO Person Detection mAP value

[thgbe]

I tried to recreate the YOLO person-detection model by following yolo_person_detection.ipynb. According to the model zoo page, it should reach a mAP@0.5 of 51.49 %. However, when I run the accuracy evaluation, this is the output:

 detections_count = 70741, unique_truth_count = 10777  
class_id = 0, name = person, ap = 21.04%     (TP = 2916, FP = 7206) 

 for conf_thresh = 0.25, precision = 0.29, recall = 0.27, F1-score = 0.28 
 for conf_thresh = 0.25, TP = 2916, FP = 7206, FN = 7861, average IoU = 19.79 % 

 IoU threshold = 50 %, used 101 Recall-points 
 mean average precision (mAP@0.50) = 0.210397, or 21.04 % 
Total Detection Time: 3 Seconds

The resulting mAP@0.5 is 21.04 %. The result is similar whether or not I train from scratch or use the pretrained model. Am I mixing anything up here?

[efxmfaiz]

Hi @thgbe ,

Thanks for testing out the yolo training flow! Initially, the evaluation is done directly in Darknet.

However, since the quantized TFLite model is being used for device implementation, the evaluation should be done at python level instead of Darknet (C++) level.

I have updated the training flow to use the developed python evaluation script. The numbers in the README has been updated properly with the evaluation accuracy results.

I will be closing this issue. Thanks !