search

kujason / avod

Code for 3D object detection for autonomous driving
MIT License
939 stars 347 forks source link

conflict on evaluation of checkpoints #166

Closed RMobina closed 4 years ago

RMobina commented 4 years ago

Thank you for releasing your code.

To evaluate the checkpoint, I run the following command: ./evaluate_object_3d_offline /KITTI/training/label_2 /avod/avod/data/outputs/avod_cars_example/predictions/kitti_predictions_3d/val/0.1/120000

which gives me the following values on the validation set for 120000: car_detection AP: 89.792343 87.403046 80.068588 car_orientation AP: 89.574364 86.905540 79.538803 car_detection_BEV AP: 89.286240 86.566772 79.359894 car_heading_BEV AP: 89.045128 86.043510 78.811020 car_detection_3D AP: 82.667107 73.233963 67.157646 car_heading_3D AP: 82.471184 72.890289 66.800842

but when I open pyramid_cars_with_aug_example_results_0.1.txt file, I got the following values for 120000

120000 done. car_detection AP: 89.953476 87.513794 80.201759 car_detection_BEV AP: 89.485191 86.658325 79.485771 car_heading_BEV AP: 89.258759 86.132568 78.931717 car_detection_3D AP: 82.731720 73.415627 67.322678 car_heading_3D AP: 82.558540 73.069168 66.963486

Could you please explain the reason of this difference between values? Because I want to know which checkpoint has the best result.

Great thanks for your help.

kujason commented 4 years ago

There may be some slight differences in one version of the saving script that discards boxes that are truncated.