search

hustvl / VAD

[ICCV 2023] VAD: Vectorized Scene Representation for Efficient Autonomous Driving
https://arxiv.org/abs/2303.12077
Apache License 2.0
498 stars 45 forks source link

Unable to reproduce the results obtained in the paper #56

Open bazyagami opened 2 months ago

bazyagami commented 2 months ago

Hello, I am currently trying to reproduce the results of VAD_tiny version and I'm unable to do so. Here's the results am getting : -------------- Motion Prediction -------------- EPA_car: -0.04155641348197631 EPA_pedestrian: -1.1379246161005119 ADE_car: 2.906290054321289 ADE_pedestrian: 1.791144847869873 FDE_car: 4.5987725257873535 FDE_pedestrian: 2.769331693649292 MR_car: 0.2698412698412698 MR_pedestrian: 0.6610169491525424

-------------- Planning -------------- gt_car:4.503418636452432 gt_pedestrian:2.099042781793319 cnt_ade_car:0.06583317054112131 cnt_ade_pedestrian:0.03731197499511623 cnt_fde_car:0.06153545614377808 cnt_fde_pedestrian:0.034577065833170544 hit_car:0.044930650517679235 hit_pedestrian:0.011721039265481539 fp_car:0.44149247899980465 fp_pedestrian:3.3787849189294783 ADE_car:0.20511303842067719 ADE_pedestrian:0.06863386929035187 FDE_car:0.2829875648021698 FDE_pedestrian:0.09575536847114563 MR_car:0.016604805626098847 MR_pedestrian:0.022856026567689 plan_L2_1s:2.202318344513195 plan_L2_2s:3.6597831307854727 plan_L2_3s:5.149097903307153 plan_obj_col_1s:0.0 plan_obj_col_2s:9.767532721234616e-05 plan_obj_col_3s:0.0001953506602465955 plan_obj_box_col_1s:0.004493065051767923 plan_obj_box_col_2s:0.01684899394412971 plan_obj_box_col_3s:0.03477241705231984 fut_valid_flag:1.0

I have followed the similar commands as given in the docs and the code was working all fine! the only problem that is occurring is with respect to the recreation of the results using the pre-trained checkpoints.

The command used to run the eval : CUDA_VISIBLE_DEVICES=0 python /mnt/work/VAD/VAD/tools/test.py /mnt/work/VAD/VAD/projects/configs/VAD/VAD_tiny_e2e.py /mnt/work/VAD/VAD/VAD_tiny.pth --launcher none --eval bbox --tmpdir tmp

hyzhou404 commented 1 month ago

Hi, have you solve the problem? I am facing the same problem

Fengtao22 commented 1 month ago

Check #18. Change: img_norm_cfg = dict( mean=[103.530, 116.280, 123.675], std=[1.0, 1.0, 1.0], to_rgb=False)

hyzhou404 commented 1 month ago

Check #18. Change: img_norm_cfg = dict( mean=[103.530, 116.280, 123.675], std=[1.0, 1.0, 1.0], to_rgb=False)

Thanks a lot!