search

mit-han-lab / bevfusion

[ICRA'23] BEVFusion: Multi-Task Multi-Sensor Fusion with Unified Bird's-Eye View Representation
https://bevfusion.mit.edu
Apache License 2.0
2.37k stars 427 forks source link

The mAP and NDS are lower than the offical. #605

Closed wyf0414 closed 8 months ago

wyf0414 commented 8 months ago

I trained 2 models.

  1. The first: epoch=12, lr=2e-4

mAP: 0.5810 mATE: 0.3097 mASE: 0.2660 mAOE: 0.3685 mAVE: 0.2897 mAAE: 0.1759 NDS: 0.6495 Eval time: 123.4s

Per-class results: Object Class AP ATE ASE AOE AVE AAE car 0.854 0.178 0.153 0.091 0.290 0.182 truck 0.567 0.325 0.196 0.094 0.283 0.225 bus 0.704 0.333 0.180 0.056 0.460 0.216 trailer 0.276 0.693 0.257 0.866 0.281 0.141 construction_vehicle 0.193 0.735 0.474 1.098 0.103 0.336 pedestrian 0.849 0.134 0.288 0.361 0.217 0.078 motorcycle 0.637 0.202 0.251 0.299 0.457 0.220 bicycle 0.420 0.172 0.251 0.378 0.226 0.010 traffic_cone 0.663 0.128 0.338 nan nan nan barrier 0.645 0.196 0.271 0.073 nan nan image

  1. The second: epoch24, lr=1e-4

mAP: 0.5771 mATE: 0.2953 mASE: 0.2738 mAOE: 0.3157 mAVE: 0.2821 mAAE: 0.1871 NDS: 0.6532 Eval time: 120.4s

Per-class results: Object Class AP ATE ASE AOE AVE AAE car 0.853 0.177 0.161 0.091 0.296 0.189 truck 0.566 0.325 0.200 0.087 0.262 0.221 bus 0.691 0.331 0.195 0.061 0.454 0.238 trailer 0.290 0.620 0.250 0.696 0.275 0.215 construction_vehicle 0.203 0.699 0.469 0.855 0.100 0.326 pedestrian 0.841 0.131 0.299 0.328 0.212 0.075 motorcycle 0.621 0.195 0.264 0.243 0.448 0.224 bicycle 0.412 0.163 0.275 0.400 0.210 0.010 traffic_cone 0.660 0.124 0.332 nan nan nan barrier 0.635 0.187 0.292 0.081 nan nan image

  1. official

mAP: 0.6854
mATE: 0.2868 mASE: 0.2538 mAOE: 0.3009 mAVE: 0.2546 mAAE: 0.1869 NDS: 0.7144 Eval time: 112.7s

Per-class results: Object Class AP ATE ASE AOE AVE AAE
car 0.892 0.170 0.148 0.060 0.272 0.185 truck 0.646 0.327 0.181 0.092 0.247 0.217 bus 0.754 0.338 0.188 0.065 0.436 0.272 trailer 0.425 0.519 0.201 0.624 0.209 0.140 construction_vehicle 0.303 0.727 0.433 0.802 0.117 0.295 pedestrian 0.882 0.134 0.288 0.386 0.217 0.100 motorcycle 0.786 0.185 0.248 0.217 0.348 0.272 bicycle 0.651 0.168 0.256 0.407 0.189 0.014 traffic_cone 0.795 0.123 0.317 nan nan nan
barrier 0.720 0.178 0.277 0.054 nan nan

It's strange that both two trains didn't converge, and 12 more epochs didn't bring performance improvement. I want to know the reason. Are the epochs not enough?

wyf0414 commented 8 months ago

@songhan @zhijian-liu Hope for your help.

Li-Whasaka commented 7 months ago

Hi, I meet the same problem, have you solved?