Does evaluation API support OBB

lfurtadoh commented 1 week ago

Search before asking

[X] I have searched the Supervision issues and found no similar feature requests.

Question

Hello! I am working with YOLOv8 for an object detection issue. I need to evaluate my model on a testset with a custom filtering method, and would like to be able to use supervision to do things like calculate mAP50 and create a confusion matrix, with the MeanAveragePrecision and ConfusionMatrix classes. I saw in issue #1227 that support for Yolo OBB dataset format is getting added.

If i do this:

dataset = sv.DetectionDataset.from_yolo(... is_obb = True)

model = YOLO(...)
    def callback(image: np.ndarray) -> sv.Detections:
    result = model(image)[0]
    return sv.Detections.from_ultralytics(result)

mean_average_precision = sv.MeanAveragePrecision.benchmark(
    dataset = dataset,
    callback = callback
)

Will it work properly, considering the dataset and detections are OBB's? Or is it still something to be added?

Additional

No response

LinasKo commented 1 week ago

Hi @lfurtadoh 👋

Looking at the code, MeanAveragePrecision does not support OBB - only standard object detection boxes. Given we're expanding our OBB support, I'll open a feature request for the community.

If you're willing, feel free to help us out!

LinasKo commented 1 week ago

I looked into it deeper. In my opinion, the cleanest solution uses shapely which we don't currently have as a dependency.

@SkalskiP, let's speak about that next time we meet.

roboflow / supervision