search

AIR-THU / DAIR-V2X

Apache License 2.0
424 stars 65 forks source link

Different results on MMdetection3D and OpenPCDet for DAIR-V2X-I training #80

Open swwwu opened 8 months ago

swwwu commented 8 months ago

Hi there. I have a question about the evaluation results. I am trying to train PointPillars based on the DAIR-V2X-I dataset with MMdetection3D and OpenPCDet. I got results similar to those of the authors of DAIR-V2X while testing on MMdetection3D. (My results on MMdetection3D: Car-3D AP(R11-iou0.5): 63.58, 54.49, 54.49, DAIR-V2X paper: 63.07, 54.00, 54.01.) However, the result on OpenPCDet is much higher than that on MMdetetion3D. (My results on OpenPCDet: Car-3D AP(R11-iou0.5): 96.67, 90.05, 90.05.) I am very confused about this. Hope someone can help me find out the answer. Thank you very much.

Result on MMdetection3D: ----------- AP11 Results ------------ Car AP11@0.70, 0.70, 0.70: bbox AP11:24.1993, 19.3548, 19.5493 bev AP11:63.4588, 54.3793, 54.3846 3d AP11:63.0491, 53.9908, 53.9994 aos AP11:12.69, 10.15, 10.25 Car AP11@0.70, 0.50, 0.50: bbox AP11:24.1993, 19.3548, 19.5493 bev AP11:63.5912, 54.5022, 54.5028 3d AP11:63.5819, 54.4903, 54.4911 aos AP11:12.69, 10.15, 10.25 ----------- AP40 Results ------------ Car AP40@0.70, 0.70, 0.70: bbox AP40:22.6050, 18.6080, 18.7952 bev AP40:67.2324, 54.7689, 54.7698 3d AP40:66.6725, 51.8965, 51.8953 aos AP40:11.85, 9.75, 9.85 Car AP40@0.70, 0.50, 0.50: bbox AP40:22.6050, 18.6080, 18.7952 bev AP40:67.4173, 54.9247, 54.9263 3d AP40:67.3984, 54.9069, 54.9088 aos AP40:11.85, 9.75, 9.85

Result on OpenPCDet:

Car AP@0.70, 0.70, 0.70: bbox AP:27.9997, 24.0742, 24.2758 bev AP:96.2945, 89.6361, 89.6418 3d AP:86.9864, 78.0029, 78.0308 aos AP:14.65, 12.59, 12.70 Car AP_R40@0.70, 0.70, 0.70: bbox AP:27.4314, 23.6478, 23.8782 bev AP:97.8024, 92.4339, 92.4493 3d AP:89.8588, 81.2604, 81.2895 aos AP:14.35, 12.37, 12.49 Car AP@0.70, 0.50, 0.50: bbox AP:27.9997, 24.0742, 24.2758 bev AP:97.0797, 90.5134, 90.5106 3d AP:96.6720, 90.0508, 90.0580 aos AP:14.65, 12.59, 12.70 Car AP_R40@0.70, 0.50, 0.50: bbox AP:27.4314, 23.6478, 23.8782 bev AP:98.8514, 95.7512, 95.7589 3d AP:98.4157, 94.9547, 94.9802 aos AP:14.35, 12.37, 12.49

bbzh commented 1 week ago

Hi @swwwu, did you figure out this? Can you share your experiences?