search

pytorch / torcheval

A library that contains a rich collection of performant PyTorch model metrics, a simple interface to create new metrics, a toolkit to facilitate metric computation in distributed training and tools for PyTorch model evaluations.
https://pytorch.org/torcheval
Other
212 stars 46 forks source link

Add COCO mAP #139

Open songyuc opened 1 year ago

songyuc commented 1 year ago

🚀 The feature

COCO mAP

COCO mAP (mean average precision) is a widely used evaluation metric for object detection models, especially for the COCO dataset. Unlike the PASCAL VOC evaluation, which has a single IoU (Intersection over Union) threshold for assessing the detection model, the COCO mAP evaluator averages the mAP of 80 classes over 10 IoU thresholds from 0.5 to 0.95 with a step size of 0.05 (AP@[0.5:0.05:0.95]). This is to avoid the bias that a single threshold may induce in the evaluation metric and to provide a more complete analysis of the detection model.

Motivation, pitch

COCO mAP has an official API, which lacks maintenance and has been outdated.

Alternatives

No response

Additional context

No response

bobakfb commented 1 year ago

Hey @songyuc, thanks for this feature request! Are you proposing to work on this yourself or did you want to leave it up for someone else to grab?

Can you explain how COCO mAP works? Do you need to have COCO on your device already or does it download part of it to run?