search

waymo-research / waymo-open-dataset

Waymo Open Dataset
https://www.waymo.com/open
Other
2.73k stars 616 forks source link

Evaluation Range Setting #240

Open Zhimin-C opened 3 years ago

Zhimin-C commented 3 years ago

Hi,

I want eval my prediction res just in the front view (0 to 80 in the x-axis, -40 to 40 in the y-axis). How do I set it in waymo open dataset evaluation?

Zhimin Chen

peisun1115 commented 3 years ago

Hi Zhimin, You can either filter your ground truth and prediction before calling the metrics or modifying the metrics code to filter.

Zhimin-C commented 3 years ago

Hi Zhimin, You can either filter your ground truth and prediction before calling the metrics or modifying the metrics code to filter.

I read the compute_detection_metric_main.cc and find that // RANGE_TYPEVEHICLE[0, 30)_LEVEL_2: [mAP 0.0784186] [mAPH 0.0709105] // RANGE_TYPEVEHICLE[30, 50)_LEVEL_2: [mAP 0.133955] [mAPH 0.103888] // RANGE_TYPEVEHICLE[50, +inf)_LEVEL_2: [mAP 0.0522752] [mAPH 0.0460845] // RANGE_TYPEPEDESTRIAN[0, 30)_LEVEL_2: [mAP 0.0458333] [mAPH 2.20291e-09] // RANGE_TYPEPEDESTRIAN[30, 50)_LEVEL_2: [mAP 0.110526] [mAPH 0.0512746] // RANGE_TYPEPEDESTRIAN[50, +inf)_LEVEL_2: [mAP 0.15] [mAPH 0.074034] // RANGE_TYPESIGN[0, 30)_LEVEL_2: [mAP 0.128993] [mAPH 0.100171] // RANGE_TYPESIGN[30, 50)_LEVEL_2: [mAP 0.0354839] [mAPH 0.0171309] // RANGE_TYPESIGN[50, +inf)_LEVEL_2: [mAP 0.0164904] [mAPH 0.01548] // RANGE_TYPECYCLIST[0, 30)_LEVEL_2: [mAP 0] [mAPH 0] // RANGE_TYPECYCLIST[30, 50)_LEVEL_2: [mAP 0] [mAPH 0] // RANGE_TYPECYCLIST[50, +inf)_LEVEL_2: [mAP 0] [mAPH 0]

Does it mean that it only counts the front view object or the [0, 30) here mean radius? If so, may you give me some suggestions about where to change the range? I am not very familiar with C++. Thanks.

Zhimin Chen