Different evaluation results when not using coco_metric

[buyukkanber]

Before Asking

• [X] I have read the README carefully. 我已经仔细阅读了README上的操作指引。

• [X] I want to train my custom dataset, and I have read the tutorials for training your custom data carefully and organize my dataset correctly; (FYI: We recommand you to apply the config files of xx_finetune.py.) 我想训练自定义数据集，我已经仔细阅读了训练自定义数据的教程，以及按照正确的目录结构存放数据集。（FYI: 我们推荐使用xx_finetune.py等配置文件训练自定义数据集。）

• [X] I have pulled the latest code of main branch to run again and the problem still existed. 我已经拉取了主分支上最新的代码，重新运行之后，问题仍不能解决。

Search before asking

• [X] I have searched the YOLOv6 issues and found no similar questions.

Question

Hello!

When I evaluate the val set, based on my trained (single class custom dataset) final model weight, with argparsers set as '--do_coco_metric=True' and '--do_pr_metric=False', I get the result below

Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.713
Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=100 ] = 0.846

However, when I change it as the following , '--do_coco_metric=False', and ,'--do_pr_metric= True', I get the result below

  P@.5iou     R@.5iou      mAP@.5     mAP@.5:.95
    0.972      0.944        0.962       0.8

There is a quite difference between 0.713 and 0.8 scores. Which one is the correct result? Could you please elaborate this change on metrics scores?

Additional

No response

[Chilicyy]

Hi @buyukkanber , do_coco_metric means using pycocotools to evaluate metrics, which is common applied in papers and competitions, while do_pr_metric means using evaluation scripts similar to yolov5 and yolov7, which you can compare the results among some other methods in convenience.