search

open-mmlab / OpenPCDet

OpenPCDet Toolbox for LiDAR-based 3D Object Detection.
Apache License 2.0
4.66k stars 1.3k forks source link

[question] Is your pretrained model using road_plane? #186

Closed Xiangzhaohong closed 4 years ago

Xiangzhaohong commented 4 years ago

I trained my own pointpillars model ,but the result is lower than the pretrained model you provided. this is my result: bbox AP:89.8853, 79.0921, 78.3211 bev AP:88.0562, 78.3999, 77.4981 3d AP:76.3732, 63.8497, 57.8995 aos AP:89.10, 77.38, 75.72 Car AP_R40@0.70, 0.70, 0.70: bbox AP:94.0155, 81.7137, 79.5236 bev AP:90.2189, 81.3183, 78.7302 3d AP:78.9282, 62.9952, 58.7199 aos AP:93.09, 79.77, 76.67 Car AP@0.70, 0.50, 0.50: bbox AP:89.8853, 79.0921, 78.3211 bev AP:90.3763, 88.6769, 88.1509 3d AP:90.1744, 82.2846, 79.3770 aos AP:89.10, 77.38, 75.72 Car AP_R40@0.70, 0.50, 0.50: bbox AP:94.0155, 81.7137, 79.5236 bev AP:95.0698, 92.3552, 90.3404 3d AP:94.4822, 84.8824, 82.6492 aos AP:93.09, 79.77, 76.67 Pedestrian AP@0.50, 0.50, 0.50: bbox AP:1.1364, 1.1364, 1.1364 bev AP:45.6964, 42.0236, 39.0739 3d AP:0.0000, 0.0000, 0.0000 aos AP:0.10, 0.10, 0.10 Pedestrian AP_R40@0.50, 0.50, 0.50: bbox AP:0.0586, 0.0422, 0.0446 bev AP:43.6153, 39.1793, 36.4527 3d AP:0.0000, 0.0000, 0.0000 aos AP:0.02, 0.02, 0.02 Pedestrian AP@0.50, 0.25, 0.25: bbox AP:1.1364, 1.1364, 1.1364 bev AP:59.3980, 55.4559, 52.3975 3d AP:12.8265, 12.3168, 12.0279 aos AP:0.10, 0.10, 0.10 Pedestrian AP_R40@0.50, 0.25, 0.25: bbox AP:0.0586, 0.0422, 0.0446 bev AP:59.3025, 54.4524, 51.1187 3d AP:11.1556, 10.5176, 9.9308 aos AP:0.02, 0.02, 0.02 Cyclist AP@0.50, 0.50, 0.50: bbox AP:1.2987, 1.1364, 1.1364 bev AP:51.7580, 38.3240, 35.4957 3d AP:0.0000, 0.0000, 0.0000 aos AP:0.59, 0.51, 0.51 Cyclist AP_R40@0.50, 0.50, 0.50: bbox AP:0.0663, 0.0063, 0.0129 bev AP:50.8476, 35.2073, 32.2254 3d AP:0.0000, 0.0000, 0.0000 aos AP:0.03, 0.00, 0.01 Cyclist AP@0.50, 0.25, 0.25: bbox AP:1.2987, 1.1364, 1.1364 bev AP:58.4693, 44.4801, 41.7354 3d AP:21.0257, 14.5286, 13.9495 aos AP:0.59, 0.51, 0.51 Cyclist AP_R40@0.50, 0.25, 0.25: bbox AP:0.0663, 0.0063, 0.0129 bev AP:58.1714, 42.6434, 39.2664 3d AP:19.4727, 12.5469, 11.6399 aos AP:0.03, 0.00, 0.01

this is the provided model's result: bbox AP:90.7786, 89.8062, 88.7936 bev AP:89.6590, 87.1725, 84.3762 3d AP:86.4617, 77.2839, 74.6530 aos AP:90.77, 89.61, 88.47 Car AP_R40@0.70, 0.70, 0.70: bbox AP:95.6607, 92.2403, 91.3167 bev AP:92.0399, 88.0556, 86.6625 3d AP:87.7518, 78.3964, 75.1843 aos AP:95.64, 92.03, 90.97 Car AP@0.70, 0.50, 0.50: bbox AP:90.7786, 89.8062, 88.7936 bev AP:90.7894, 90.1848, 89.4635 3d AP:90.7894, 90.0675, 89.2495 aos AP:90.77, 89.61, 88.47 Car AP_R40@0.70, 0.50, 0.50: bbox AP:95.6607, 92.2403, 91.3167 bev AP:95.6987, 94.7077, 93.9983 3d AP:95.6874, 94.3709, 93.4244 aos AP:95.64, 92.03, 90.97 Pedestrian AP@0.50, 0.50, 0.50: bbox AP:66.5436, 62.4922, 59.3026 bev AP:61.6348, 56.2747, 52.6007 3d AP:57.7500, 52.2916, 47.9072 aos AP:48.63, 45.62, 42.93 Pedestrian AP_R40@0.50, 0.50, 0.50: bbox AP:66.5852, 62.4351, 58.8016 bev AP:61.5971, 56.0143, 52.0457 3d AP:57.3015, 51.4145, 46.8715 aos AP:45.89, 42.99, 40.03 Pedestrian AP@0.50, 0.25, 0.25: bbox AP:66.5436, 62.4922, 59.3026 bev AP:72.5064, 69.5191, 66.4626 3d AP:72.4368, 69.3244, 65.3180 aos AP:48.63, 45.62, 42.93 Pedestrian AP_R40@0.50, 0.25, 0.25: bbox AP:66.5852, 62.4351, 58.8016 bev AP:73.8776, 70.4969, 66.6494 3d AP:73.7943, 70.2258, 66.0435 aos AP:45.89, 42.99, 40.03 Cyclist AP@0.50, 0.50, 0.50: bbox AP:85.2661, 72.9744, 68.9914 bev AP:82.2716, 66.2565, 62.6405 3d AP:80.0568, 62.6873, 59.7069 aos AP:84.72, 71.09, 67.13 Cyclist AP_R40@0.50, 0.50, 0.50: bbox AP:88.5723, 74.0385, 69.8009 bev AP:85.2713, 66.3460, 62.3646 3d AP:81.5782, 62.9381, 58.9814 aos AP:87.91, 71.98, 67.81 Cyclist AP@0.50, 0.25, 0.25: bbox AP:85.2661, 72.9744, 68.9914 bev AP:86.6035, 70.6055, 66.9244 3d AP:86.6035, 70.6055, 66.9244 aos AP:84.72, 71.09, 67.13 Cyclist AP_R40@0.50, 0.25, 0.25: bbox AP:88.5723, 74.0385, 69.8009 bev AP:88.8812, 71.7453, 67.7714 3d AP:88.8812, 71.7453, 67.7714

Thanks!!!

sshaoshuai commented 4 years ago

Yes, the pretrained model is trained with road plane. But the effects of road planes is minor. The performance should be similar without road planes. Could you post your training command here?

Xiangzhaohong commented 4 years ago

@sshaoshuai hi, this is my command bash scripts/dist_train.sh 2 --cfg_file cfgs/kitti_models/pointpillar.yaml --batch_size 14 --epochs 80 I have two 2080ti

sshaoshuai commented 4 years ago

Could you try to train with batch_size 8 on two GPUs? Although I think the batch_size 14 should also be ok, I am not sure whether a large batch_size will have negative effects on this line: https://github.com/open-mmlab/OpenPCDet/blob/master/pcdet/models/backbones_3d/vfe/pillar_vfe.py#L29-L36

Xiangzhaohong commented 4 years ago

@sshaoshuai OK , I will try , thanks a lot for your code !

Xiangzhaohong commented 4 years ago

@sshaoshuai I have trained with batch size 8 , and the result is the same as before. I just found that I use pcdet0.1's data info , and didn't set the DATABASE_WITH_FAKELIDAR option in tools/cfgs/dataset_configs/kitti_dataset.yaml as True. I will try to use pcdet0.2'data info, and evaluate the result.

Xiangzhaohong commented 4 years ago

@sshaoshuai I solved this problem, it's the reason of data info. Thanks a lot!!