Yzichen / FlashOCC

229 stars 27 forks source link

inconsistency between the performance of github #45

Open Rayn-Wu opened 2 months ago

Rayn-Wu commented 2 months ago

Hi , when I test the config of flashocc-r50-4d-stereo, the result is 38.07 :

===> per class IoU of 6019 samples:
===> others - IoU = 9.37
===> barrier - IoU = 46.72
===> bicycle - IoU = 19.43
===> bus - IoU = 41.69
===> car - IoU = 51.07
===> construction_vehicle - IoU = 24.99
===> motorcycle - IoU = 21.24
===> pedestrian - IoU = 23.6
===> traffic_cone - IoU = 24.22
===> trailer - IoU = 31.21
===> truck - IoU = 38.18
===> driveable_surface - IoU = 81.12
===> other_flat - IoU = 40.13
===> sidewalk - IoU = 51.93
===> terrain - IoU = 55.31
===> manmade - IoU = 47.01
===> vegetation - IoU = 39.89
===> mIoU of 6019 samples: 38.07
{'mIoU': array([0.094, 0.467, 0.194, 0.417, 0.511, 0.25 , 0.212, 0.236, 0.242,
       0.312, 0.382, 0.811, 0.401, 0.519, 0.553, 0.47 , 0.399, 0.903])}

which is a little higher than the result you provide:

 with_pretrain:
 ===> per class IoU of 6019 samples:
 ===> others - IoU = 9.08
 ===> barrier - IoU = 46.32
 ===> bicycle - IoU = 17.71
 ===> bus - IoU = 42.7
 ===> car - IoU = 50.64
 ===> construction_vehicle - IoU = 23.72
 ===> motorcycle - IoU = 20.13
 ===> pedestrian - IoU = 22.34
 ===> traffic_cone - IoU = 24.09
 ===> trailer - IoU = 30.26
 ===> truck - IoU = 37.39
 ===> driveable_surface - IoU = 81.68
 ===> other_flat - IoU = 40.13
 ===> sidewalk - IoU = 52.34
 ===> terrain - IoU = 56.46
 ===> manmade - IoU = 47.69
 ===> vegetation - IoU = 40.6
 ===> mIoU of 6019 samples: 37.84 with_pretrain:
 ===> per class IoU of 6019 samples:

BTW, I use 4090 for testing. Could the gap caused by this?

Rayn-Wu commented 2 months ago

I use this model in readme for testing, and find the mIoU gap above : https://github.com/Yzichen/FlashOCC/blob/master/projects/configs/flashocc/flashocc-r50-4d-stereo.py