mAP calculation

[cadwallader113]

I have a question about mAP which in evaluate.py.

for i in range(0, len(score_thresholds), 4): print(f"{i}, {precisions[i]}") // add sums_AP += precisions[i] mAP = sums_AP / 11 * 100

It seems precisions length is not match 11 sample. I add the print then confirmed the precisions[i].

==precisions== 0, 1.0 4, 0.875968992248062 8, 0.8194945848375451 12, 0.776255707762557 16, 0.7104851330203443 20, 0.648 24, 0.522273425499232 28, 0.308589195491644 Sometime it takes 8~10samples.

Is the following implementation inconvenient? for i in range(0, len(score_thresholds)): print(f"{i}, {precisions[i]}") // add sums_AP += precisions[i] mAP = sums_AP / len(precisions) * 100

[zhulf0804]

Hi,

• Please refer to https://github.com/zhulf0804/PointPillars/issues/6#issue-1321816729 for the first question.
• Convenient. However, to keep consistent with mmdet3d, we calculate mAP at 11 recall positions. In addition, the denominator is not len(precisions). It should be recall sample interval (1/40 -> 40) as shown in the following PR-Curve figure.

[cadwallader113]

Thank you for your reply. The idea of being consistent with mmdet3d made sense to me. I also understand that the total number is not 41 (issue6) because the maximum score_threshold does not reach recall=1.0. I reconsidered that mAP is calculated by assuming score_threshold is 0.0 when score_threshold is not present (eg recall is 0.9, 1.0). mAP=(1.0+0.8+0.8+0.7+0.7+0.6+0.5+0.3+0.0+0.0+0.0)/11 Do you agree with this understanding?

[zhulf0804]

Yes.

The value of longitudinal coordinate (precision) is considered to be 0.0.