Clarification on ablation experiments

[LokeshJatangi]

Hi @outsidercsy ,

We wanted to understand the contribution of different modules in VAD. For the same, in the config VAD_base_e2e.py, we updated the loss-weight = 0.0 for all losses except the loss_plan_reg and trained the model. With this limited supervision, we obtained the following planning results at 5th Epoch and were a bit surprised.

We were wondering if these planning results (which are better than the results reported in Table 1) are expected or we are missing something in interpreting these results? Should we look more closely in the collision metrics?

Could it be possible that a pretrained BEVFormer checkpoint is getting loaded into VAD model ?

Any insights/pointers would be super helpful.

Planning results :-

plan_L2_1s:0.38330350474705394
plan_L2_2s:0.6309658567030366
plan_L2_3s:0.9622495931838358
plan_obj_col_1s:0.0
plan_obj_col_2s:0.0
plan_obj_col_3s:0.0
plan_obj_box_col_1s:0.0038093377612815004
plan_obj_box_col_2s:0.005860519632740769
plan_obj_box_col_3s:0.008693104247069728

Thanks in advance !