open-mmlab / mmdetection

OpenMMLab Detection Toolbox and Benchmark
https://mmdetection.readthedocs.io
Apache License 2.0
29.13k stars 9.38k forks source link

What is the difference of the bbox_map of coco and map of voc? #7361

Open yichuangzhang opened 2 years ago

yichuangzhang commented 2 years ago

Thanks for your error report and we appreciate it a lot.

Checklist

  1. I have searched related issues but cannot get the expected help.
  2. I have read the FAQ documentation but cannot get the expected help.
  3. The bug has not been fixed in the latest version.

Describe the bug A clear and concise description of what the bug is.

Reproduction

  1. What command or script did you run?
A placeholder for the command.
  1. Did you make any modifications on the code or config? Did you understand what you have modified?
  2. What dataset did you use?

Environment

  1. Please run python mmdet/utils/collect_env.py to collect necessary environment information and paste it here.
  2. You may add addition that may be helpful for locating the problem, such as
    • How you installed PyTorch [e.g., pip, conda, source]
    • Other environment variables that may be related (such as $PATH, $LD_LIBRARY_PATH, $PYTHONPATH, etc.)

Error traceback If applicable, paste the error trackback here.

A placeholder for trackback.

Bug fix If you have already identified the reason, you can provide the information here. If you are willing to create a PR to fix it, please also leave a comment here and that would be much appreciated!

BIGWangYuDong commented 2 years ago

COCO evaluation metric bbox is used to evaluate coco dataset, which set the iou_thr = [0.5, 0.95, 0.05]. It used pycocotools to evaluate the result. VOC evaluation metric mAP set iou_thr=0.5 as default and using 11points method to calculate AP. The evaluation code is at mmdet/core/evaluation/mean_ap.py

For more details, you can have a look at the dataset.evaluate code in mmdet/datasets such as: https://github.com/open-mmlab/mmdetection/blob/70f6d9cfade4a2f0b198e4f64776521d181b28be/mmdet/datasets/voc.py#L34-L41 and https://github.com/open-mmlab/mmdetection/blob/70f6d9cfade4a2f0b198e4f64776521d181b28be/mmdet/datasets/coco.py#L386-L395