search

zhulf0804 / PointPillars

A Simple PointPillars PyTorch Implementation for 3D LiDAR(KITTI) Detection.
MIT License
454 stars 112 forks source link

Question about mAP calculation #51

Closed jonasdieker closed 1 year ago

jonasdieker commented 1 year ago

Hi @zhulf0804, hope you are doing well!

I have read #6 and #12 already, but I still have some questions regarding the evaluation file. I also understand that you are trying to be consistent with mmdet3d, but perhaps you have still have some insights into why specific things were done the way they were:

  1. Why are 41 (or less) score_thresholds used but only every fourth one is used to calculate the AUC?
  2. It seems that the recalls are not used at all when computing the AP. Only the precisions are summed up and divided by how many values were summed (11, assuming 41 thresholds). Is this why the thresholds are computed first, in order to ensure that the recall steps are even in size?
zhulf0804 commented 1 year ago

Hello @jonasdieker. For the first question, I think 11 score_thresholds is also ok, but 41 is more precise. More details please refer to KITTI Det Dataset,

image

For the second question, Yes. And AP means average precision under different recalls. Best.

jonasdieker commented 1 year ago

Great, thank you!