search

tusen-ai / SST

Code for a series of work in LiDAR perception, including SST (CVPR 22), FSD (NeurIPS 22), FSD++ (TPAMI 23), FSDv2, and CTRL (ICCV 23, oral).
Apache License 2.0
801 stars 102 forks source link

error performance when evaluation #175

Open niranyingluofen opened 11 months ago

niranyingluofen commented 11 months ago

Hi~ I tried to train fsd on waymo perception，and get the performance on validation set as follows. They are all 0. OBJECT_TYPE_TYPE_VEHICLE_LEVEL_1: [mAP 0.749648] [mAPH 0] OBJECT_TYPE_TYPE_VEHICLE_LEVEL_2: [mAP 0.669701] [mAPH 0] OBJECT_TYPE_TYPE_PEDESTRIAN_LEVEL_1: [mAP 0.80301] [mAPH 0] OBJECT_TYPE_TYPE_PEDESTRIAN_LEVEL_2: [mAP 0.73128] [mAPH 0] OBJECT_TYPE_TYPE_SIGN_LEVEL_1: [mAP 0] [mAPH 0] OBJECT_TYPE_TYPE_SIGN_LEVEL_2: [mAP 0] [mAPH 0] OBJECT_TYPE_TYPE_CYCLIST_LEVEL_1: [mAP 0.744445] [mAPH 0] OBJECT_TYPE_TYPE_CYCLIST_LEVEL_2: [mAP 0.719278] [mAPH 0]

My command is

DIR=fsd WORK=work_dirs CONFIG=fsd_waymoD1_1x bash ./tools/dist_test.sh configs/work_dirs/$CONFIG.py ./work_dirs/$CONFIG/latest.pth 8 --eval fast --options "pklfile_prefix=./work_dirs/$CONFIG/results"

Abyssaledge commented 11 months ago

I think there is something wrong with your evaluation tool or ground-truth file if you still get 0 using the provided result.bin files. How do you build the evaluation tool and get the ground-truth file?