The result of Centerpoint training on KITTI.

[yaxi333]

Hi, I am now trying to study how to training Centerpoint on KITTI. My setting is training on the original KITTI dataset of 20 epoches. And what I got is as follows:

Car AP@0.70, 0.70, 0.70: bbox AP:94.7996, 88.9117, 87.8464 bev AP:89.7246, 86.8356, 83.5177 3d AP:86.4783, 76.2949, 72.4491 aos AP:94.75, 88.76, 87.63 Car AP_R40@0.70, 0.70, 0.70: bbox AP:96.9610, 91.3264, 88.8810 bev AP:93.9845, 87.7600, 85.2048 3d AP:88.2733, 75.9294, 73.1899 aos AP:96.92, 91.16, 88.66 Car AP@0.70, 0.50, 0.50: bbox AP:94.7996, 88.9117, 87.8464 bev AP:94.9225, 89.3233, 88.8077 3d AP:94.8194, 89.2578, 88.6416 aos AP:94.75, 88.76, 87.63 Car AP_R40@0.70, 0.50, 0.50: bbox AP:96.9610, 91.3264, 88.8810 bev AP:97.2669, 93.8379, 93.0098 3d AP:97.1964, 93.4531, 91.2541 aos AP:96.92, 91.16, 88.66 Pedestrian AP@0.50, 0.50, 0.50: bbox AP:66.8845, 65.2675, 62.3663 bev AP:57.1777, 53.5311, 50.7458 3d AP:50.1384, 48.6857, 44.5479 aos AP:64.70, 62.75, 59.74 Pedestrian AP_R40@0.50, 0.50, 0.50: bbox AP:68.0557, 65.3659, 62.4463 bev AP:55.0341, 52.0861, 48.6433 3d AP:48.6057, 45.7464, 41.7122 aos AP:65.56, 62.53, 59.48 Pedestrian AP@0.50, 0.25, 0.25: bbox AP:66.8845, 65.2675, 62.3663 bev AP:74.2538, 72.4498, 69.3171 3d AP:74.1821, 72.2494, 69.1611 aos AP:64.70, 62.75, 59.74 Pedestrian AP_R40@0.50, 0.25, 0.25: bbox AP:68.0557, 65.3659, 62.4463 bev AP:74.8110, 73.2887, 70.1591 3d AP:74.7206, 73.0705, 69.9633 aos AP:65.56, 62.53, 59.48 Cyclist AP@0.50, 0.50, 0.50: bbox AP:84.2834, 73.6666, 70.5905 bev AP:80.0970, 66.7770, 63.7807 3d AP:74.6868, 62.2825, 57.8456 aos AP:84.12, 73.11, 70.07 Cyclist AP_R40@0.50, 0.50, 0.50: bbox AP:88.0991, 74.7699, 71.1338 bev AP:81.6918, 67.8036, 63.8179 3d AP:75.0102, 61.3894, 57.8612 aos AP:87.91, 74.20, 70.57 Cyclist AP@0.50, 0.25, 0.25: bbox AP:84.2834, 73.6666, 70.5905 bev AP:81.9637, 69.6221, 67.1479 3d AP:81.9637, 69.6218, 67.1479 aos AP:84.12, 73.11, 70.07 Cyclist AP_R40@0.50, 0.25, 0.25: bbox AP:88.0991, 74.7699, 71.1338 bev AP:85.6864, 70.9843, 67.4068 3d AP:85.6864, 70.9842, 67.4062 aos AP:87.91, 74.20, 70.57

My questions are:

1. I am not sure if it's right or not? Did you get similar results?
2. And what does the "Cyclist AP_R40@0.50, 0.25, 0.25" mean? Could we modify the setting? Or this setting is fixed because we use the KITTI dataset?

Thanks in advance!

[tianweiy]

sounds reasonable. Some other people also tried this code and he got the following result with 80 epochs.

bbox AP:95.4993, 89.6682, 88.9629
bev  AP:89.8679, 87.6789, 85.9437
3d   AP:88.2657, 78.2677, 76.8833
aos  AP:95.46, 89.56, 88.80
Car AP_R40@0.70, 0.70, 0.70:
bbox AP:97.6554, 94.1077, 91.6578
bev  AP:94.0013, 88.4244, 87.5006
3d   AP:90.4481, 81.1609, 76.7102
aos  AP:97.62, 93.97, 91.47
Car AP@0.70, 0.50, 0.50:
bbox AP:95.4993, 89.6682, 88.9629
bev  AP:95.5043, 89.8126, 89.2684
3d   AP:95.4466, 89.7798, 89.1959
aos  AP:95.46, 89.56, 88.80
Car AP_R40@0.70, 0.50, 0.50:
bbox AP:97.6554, 94.1077, 91.6578
bev  AP:97.7235, 94.6102, 94.0849
3d   AP:97.5920, 94.5400, 93.8753
aos  AP:97.62, 93.97, 91.47
Pedestrian AP@0.50, 0.50, 0.50:
bbox AP:72.7394, 69.3657, 66.4774
bev  AP:57.3676, 54.3496, 51.8871
3d   AP:52.3187, 50.2359, 45.8078
aos  AP:71.13, 67.24, 64.13
Pedestrian AP_R40@0.50, 0.50, 0.50:
bbox AP:72.9846, 70.1345, 66.9384
bev  AP:57.4010, 55.0816, 51.4492
3d   AP:52.3284, 49.7867, 45.8620
aos  AP:71.35, 67.77, 64.27
Pedestrian AP@0.50, 0.25, 0.25:
bbox AP:72.7394, 69.3657, 66.4774
bev  AP:76.5806, 75.6884, 72.9038
3d   AP:76.5774, 75.4502, 72.5838
aos  AP:71.13, 67.24, 64.13
Pedestrian AP_R40@0.50, 0.25, 0.25:
bbox AP:72.9846, 70.1345, 66.9384
bev  AP:78.2667, 77.0423, 73.7477
3d   AP:78.2629, 76.7177, 73.4710
aos  AP:71.35, 67.77, 64.27
Cyclist AP@0.50, 0.50, 0.50:
bbox AP:86.0278, 76.6100, 73.2047
bev  AP:78.2635, 66.3874, 62.1329
3d   AP:75.6480, 63.4762, 58.7180
aos  AP:85.89, 75.86, 72.50
Cyclist AP_R40@0.50, 0.50, 0.50:
bbox AP:91.0578, 77.7942, 74.1132
bev  AP:82.0086, 66.1239, 62.3801
3d   AP:77.1098, 62.2293, 58.4965
aos  AP:90.90, 76.93, 73.31
Cyclist AP@0.50, 0.25, 0.25:
bbox AP:86.0278, 76.6100, 73.2047
bev  AP:82.8018, 71.4711, 68.1855
3d   AP:82.8018, 71.4711, 68.1855
aos  AP:85.89, 75.86, 72.50
Cyclist AP_R40@0.50, 0.25, 0.25:
bbox AP:91.0578, 77.7942, 74.1132
bev  AP:87.1466, 72.2116, 68.6427
3d   AP:87.1466, 72.2116, 68.6427
aos  AP:90.90, 76.93, 73.31

And what does the "Cyclist AP_R40@0.50, 0.25, 0.25" mean? Could we modify the setting? Or this setting is fixed because we use the KITTI dataset?

I guess it means using iou threshold 0.5 / 0.25 /0.25 for easy /moderate / hard split evaluation. The evaluation code is taken from openpcdet (which is taken from second), please check the code here https://github.com/tianweiy/CenterPoint-KITTI/tree/main/pcdet/datasets/kitti/kitti_object_eval_python for modification

[yaxi333]

OK. Thank you very much!

[linchunmian]

Hi, I am now trying to study how to training Centerpoint on KITTI. My setting is training on the original KITTI dataset of 20 epoches. And what I got is as follows:

Car AP@0.70, 0.70, 0.70: bbox AP:94.7996, 88.9117, 87.8464 bev AP:89.7246, 86.8356, 83.5177 3d AP:86.4783, 76.2949, 72.4491 aos AP:94.75, 88.76, 87.63 Car AP_R40@0.70, 0.70, 0.70: bbox AP:96.9610, 91.3264, 88.8810 bev AP:93.9845, 87.7600, 85.2048 3d AP:88.2733, 75.9294, 73.1899 aos AP:96.92, 91.16, 88.66 Car AP@0.70, 0.50, 0.50: bbox AP:94.7996, 88.9117, 87.8464 bev AP:94.9225, 89.3233, 88.8077 3d AP:94.8194, 89.2578, 88.6416 aos AP:94.75, 88.76, 87.63 Car AP_R40@0.70, 0.50, 0.50: bbox AP:96.9610, 91.3264, 88.8810 bev AP:97.2669, 93.8379, 93.0098 3d AP:97.1964, 93.4531, 91.2541 aos AP:96.92, 91.16, 88.66 Pedestrian AP@0.50, 0.50, 0.50: bbox AP:66.8845, 65.2675, 62.3663 bev AP:57.1777, 53.5311, 50.7458 3d AP:50.1384, 48.6857, 44.5479 aos AP:64.70, 62.75, 59.74 Pedestrian AP_R40@0.50, 0.50, 0.50: bbox AP:68.0557, 65.3659, 62.4463 bev AP:55.0341, 52.0861, 48.6433 3d AP:48.6057, 45.7464, 41.7122 aos AP:65.56, 62.53, 59.48 Pedestrian AP@0.50, 0.25, 0.25: bbox AP:66.8845, 65.2675, 62.3663 bev AP:74.2538, 72.4498, 69.3171 3d AP:74.1821, 72.2494, 69.1611 aos AP:64.70, 62.75, 59.74 Pedestrian AP_R40@0.50, 0.25, 0.25: bbox AP:68.0557, 65.3659, 62.4463 bev AP:74.8110, 73.2887, 70.1591 3d AP:74.7206, 73.0705, 69.9633 aos AP:65.56, 62.53, 59.48 Cyclist AP@0.50, 0.50, 0.50: bbox AP:84.2834, 73.6666, 70.5905 bev AP:80.0970, 66.7770, 63.7807 3d AP:74.6868, 62.2825, 57.8456 aos AP:84.12, 73.11, 70.07 Cyclist AP_R40@0.50, 0.50, 0.50: bbox AP:88.0991, 74.7699, 71.1338 bev AP:81.6918, 67.8036, 63.8179 3d AP:75.0102, 61.3894, 57.8612 aos AP:87.91, 74.20, 70.57 Cyclist AP@0.50, 0.25, 0.25: bbox AP:84.2834, 73.6666, 70.5905 bev AP:81.9637, 69.6221, 67.1479 3d AP:81.9637, 69.6218, 67.1479 aos AP:84.12, 73.11, 70.07 Cyclist AP_R40@0.50, 0.25, 0.25: bbox AP:88.0991, 74.7699, 71.1338 bev AP:85.6864, 70.9843, 67.4068 3d AP:85.6864, 70.9842, 67.4062 aos AP:87.91, 74.20, 70.57

My questions are:

1. I am not sure if it's right or not? Did you get similar results?
2. And what does the "Cyclist AP_R40@0.50, 0.25, 0.25" mean? Could we modify the setting? Or this setting is fixed because we use the KITTI dataset?

Thanks in advance!

Hi, yaxi. There is a tricky problem when I run the CenterPoint-KITTI code as follows: I never modify any functions or implementations in this project, but the bug still exists. Did you have encountered it when you ran the code, or how to solve this problem in your view? Any instructions would be appreciated!

[yaxi333]

The error seems to be related to something about CUDA. I haven't met the error before. But I think maybe you can try to check if your CUDA successfully runs now?

[linchunmian]

The error seems to be related to something about CUDA. I haven't met the error before. But I think maybe you can try to check if your CUDA successfully runs now?

Thanks for quick reply. You know, this project is based on OpenPCDet, and I have tried to run PV-RCNN model using config file 'pvrcnn.yaml' and corresponding modules. The CUDA usage is normal and training results are OK. The most difference between CenterPoint_rcnn and PVRCNN model is the choice of 'DenseHead', the former uses 'CenterHead', and the latter is 'AnchorHeadSingle'. Maybe the definition of variances or tensors in 'centerpoint_head_single.py' file is abnormal, but I have not find the error localization yet. Anyway, thanks for your help.

[StiphyJay]

Hi，tianwei， Thanks for your great work. I have trainied on the original KITTI dataset of 80 epoches. And what I got is as follows:

2021-12-05 00:41:48,050 INFO * Performance of EPOCH 76 *** INFO:pcdet.utils.common_utils:* Performance of EPOCH 76 *** 2021-12-05 00:41:48,050 INFO Generate label finished(sec_per_example: 0.0134 second). INFO:pcdet.utils.common_utils:Generate label finished(sec_per_example: 0.0134 second). 2021-12-05 00:41:48,051 INFO recall_roi_0.3: 0.957791 INFO:pcdet.utils.common_utils:recall_roi_0.3: 0.957791 2021-12-05 00:41:48,051 INFO recall_rcnn_0.3: 0.958814 INFO:pcdet.utils.common_utils:recall_rcnn_0.3: 0.958814 2021-12-05 00:41:48,052 INFO recall_roi_0.5: 0.910461 INFO:pcdet.utils.common_utils:recall_roi_0.5: 0.910461 2021-12-05 00:41:48,052 INFO recall_rcnn_0.5: 0.917743 INFO:pcdet.utils.common_utils:recall_rcnn_0.5: 0.917743 2021-12-05 00:41:48,053 INFO recall_roi_0.7: 0.659025 INFO:pcdet.utils.common_utils:recall_roi_0.7: 0.659025 2021-12-05 00:41:48,053 INFO recall_rcnn_0.7: 0.728881 INFO:pcdet.utils.common_utils:recall_rcnn_0.7: 0.728881 2021-12-05 00:41:48,056 INFO Average predicted number of objects(3769 samples): 11.529 INFO:pcdet.utils.common_utils:Average predicted number of objects(3769 samples): 11.529 2021-12-05 00:42:12,082 INFO Car AP@0.70, 0.70, 0.70: bbox AP:96.5962, 88.8321, 88.6788 bev AP:89.1489, 86.6326, 86.1827 3d AP:86.9565, 77.0815, 76.8418 aos AP:96.55, 88.71, 88.46 Car AP_R40@0.70, 0.70, 0.70: bbox AP:98.2095, 93.5939, 91.5310 bev AP:94.0437, 87.4882, 87.4322 3d AP:89.5494, 80.1600, 78.4115 aos AP:98.17, 93.44, 91.27 Car AP@0.70, 0.50, 0.50: bbox AP:96.5962, 88.8321, 88.6788 bev AP:96.6649, 93.6752, 88.7519 3d AP:96.5549, 88.8252, 88.6973 aos AP:96.55, 88.71, 88.46 Car AP_R40@0.70, 0.50, 0.50: bbox AP:98.2095, 93.5939, 91.5310 bev AP:98.2117, 95.6037, 93.7540 3d AP:98.1664, 93.6909, 93.5949 aos AP:98.17, 93.44, 91.27 Pedestrian AP@0.50, 0.50, 0.50: bbox AP:67.8469, 64.0615, 61.9882 bev AP:63.6253, 56.8471, 53.4049 3d AP:60.2016, 53.3828, 49.2598 aos AP:65.38, 61.01, 58.23 Pedestrian AP_R40@0.50, 0.50, 0.50: bbox AP:69.1837, 64.2857, 61.7152 bev AP:63.6414, 56.6040, 52.1890 3d AP:59.6199, 52.2266, 48.1070 aos AP:66.34, 60.77, 57.53 Pedestrian AP@0.50, 0.25, 0.25: bbox AP:67.8469, 64.0615, 61.9882 bev AP:76.9900, 73.4777, 70.8975 3d AP:76.9558, 73.3793, 70.7273 aos AP:65.38, 61.01, 58.23 Pedestrian AP_R40@0.50, 0.25, 0.25: bbox AP:69.1837, 64.2857, 61.7152 bev AP:79.1741, 74.8277, 71.6753 3d AP:79.1308, 74.7097, 71.5131 aos AP:66.34, 60.77, 57.53 Cyclist AP@0.50, 0.50, 0.50: bbox AP:89.5139, 76.4873, 73.5068 bev AP:86.2640, 70.3807, 66.1857 3d AP:86.0349, 69.6714, 65.4482 aos AP:89.43, 75.98, 73.05 Cyclist AP_R40@0.50, 0.50, 0.50: bbox AP:94.4208, 76.8401, 73.8751 bev AP:88.6273, 71.7297, 67.4997 3d AP:88.9594, 69.7150, 65.4788 aos AP:94.30, 76.37, 73.37 Cyclist AP@0.50, 0.25, 0.25: bbox AP:89.5139, 76.4873, 73.5068 bev AP:88.6250, 74.1480, 70.6165 3d AP:88.6250, 74.1480, 70.6165 aos AP:89.43, 75.98, 73.05 Cyclist AP_R40@0.50, 0.25, 0.25: bbox AP:94.4208, 76.8401, 73.8751 bev AP:93.2914, 74.1941, 71.1469 3d AP:93.2914, 74.1941, 71.1469 aos AP:94.30, 76.37, 73.37

I am not sure this is the correct result or not. Should I compare it with the result you released? Or is it compared with which baseline? I am looking forward to your reply!

Thanks!

[tianweiy]

your results look reasonable. My results (a few months ago) are similar to https://github.com/tianweiy/CenterPoint-KITTI/issues/9#issuecomment-874426375 Yours seem a little lower for car but much higher for ped / cyclist. I guess they are within reasonable variance.

[yangzheng-yz]

@StiphyJay Hi, im also training centerpoint on kitti detection dataset. I have tried several different model config defined in tools/cfgs/kitti_models. My best result was generated by centerpoint_rcnn.yaml. But this result is still much lower than yours. I want to know about your train configurations, for example, the batch_size, the total epoch, and the yaml file used on training.

Thanks!