Recall and Confusion matrix, also the confusion matrix, of the WBF

moahaimen commented 9 months ago

I used the WBF for combining 3 yolov5 models, but now i need the final Precision and Recall and Confusion matrix, also the confusion matrix, of the WBF how can i do it,

bshakya77 commented 3 months ago

@moahaimen, I am using YOLOv8n, YOLOv8s, YOLOv8m, YOLOv8l, YOLOv8x models with WBF for object detection in VisDrone2019 dataset.

Did you able to perform the evaluation for the ensemble model predictions?

Your feedback is highly appreciated.

Regards, Bijay

KAKAROT12419 commented 3 months ago

can you please tell me what format did weighted box fusion is giving output. Like its xmin, ymin , xmax, ymax or anything else.

bshakya77 commented 3 months ago

can you please tell me what format did weighted box fusion is giving output. Like its xmin, ymin , xmax, ymax or anything else.

Yes, it gives output in xmin, ymin , xmax, ymax format. But, it is in de-normalized format.

Sample prediction result:

bounding box coordinates
conf scores
class label

{'0000006_01659_d_0000004': [[404, 364, 216, 94], [452, 609, 156, 72], [435, 298, 196, 85], [463, 575, 145, 56], [446, 672, 160, 66], [489, 525, 123, 55], [551, 503, 106, 54], [442, 721, 164, 43], [481, 257, 182, 72], [509, 209, 164, 61], [541, 168, 137, 54], [364, 425, 209, 103], [684, 119, 33, 55], [712, 124, 37, 51], [743, 124, 21, 50]], '0000006_05168_d_0000013': [[594, 285, 62, 72], [743, 278, 21, 73], [83, 248, 131, 58], [679, 282, 58, 70], [103, 277, 126, 60], [137, 200, 123, 51], [195, 179, 110, 47], [528, 286, 59, 64], [0, 230, 31, 60], [123, 45, 16, 25], [115, 216, 131, 58]], '0000011_06595_d_0000010': [[746, 191, 18, 55], [697, 151, 18, 45], [513, 324, 36, 62], [496, 344, 75, 47], [340, 394, 60, 75], [255, 103, 20, 45], [533, 125, 18, 46]]}
{'0000006_01659_d_0000004': [[0.86604], [0.84769], [0.81754], [0.81647], [0.81016], [0.80896], [0.76495], [0.73147], [0.6655], [0.63703], [0.62914], [0.45945], [0.44228], [0.42985], [0.40013]], '0000006_05168_d_0000013': [[0.84664], [0.81553], [0.76947], [0.7288], [0.72482], [0.64824], [0.56348], [0.44876], [0.37089], [0.22063], [0.20727]], '0000011_06595_d_0000010': [[0.68253], [0.67364], [0.54689], [0.37154], [0.35386], [0.27636], [0.24566]]}
{'0000006_01659_d_0000004': [5, 3, 5, 3, 3, 3, 3, 3, 5, 5, 5, 5, 5, 5, 5], '0000006_05168_d_0000013': [3, 3, 3, 3, 3, 3, 3, 3, 3, 1, 3], '0000011_06595_d_0000010': [6, 0, 1, 2, 2, 0, 0]}

I am trying to convert it in .csv format that is required for benchmark. I am confused with the .csv annotation file used in benchmark file (ann). Is it annotation from test data for those images used in prediction? Also, is it necessary that the prediction result (xmin, ymin , xmax, ymax) should be in normalized form?

Thank you, Bijay

KAKAROT12419 commented 3 months ago

Thanks for information Bijay. See i am not clear with benchmark file can you tell me something about benchmark file please.

bshakya77 commented 3 months ago

Thanks for information Bijay. See i am not clear with benchmark file can you tell me something about benchmark file please.

I am referring to this file: https://github.com/ZFTurbo/Weighted-Boxes-Fusion/blob/master/benchmark_oid/run_benchmark_oid.py

I see it takes the two .csv files: detection.csv and annotation.csv and calculates the mean average precision.

I wanted to know if the (xmin, ymin , xmax, ymax) should be in normalized form?
For annotation file, does the file contain the annotations of test image data from the dataset used (like those images used for predictions)?
Have you calculated any other evaluation metrics like recall and accuracy for ensemble prediction?

Thank you, Bijay

ZFTurbo / Weighted-Boxes-Fusion

Recall and Confusion matrix, also the confusion matrix, of the WBF #61