search

PJLab-ADG / 3DTrans

An open-source codebase for exploring autonomous driving pre-training
https://bobrown.github.io/Team_3DTrans.github.io/
Apache License 2.0
585 stars 72 forks source link

About evaluation of source-pretrained model. #4

Closed YannnWanggg closed 1 year ago

YannnWanggg commented 1 year ago

Thanks for your great job. I found some issues with the evaluation of source-pretrained model in GETTING_STARTED_ADA.md. I noticed that some of the yaml files don't include DATA_CONFIG_TAR. For example, I was testing with the following command

bash scripts/dist_test.sh 4 --cfg_file cfgs/DA/nusc_kitti/source_only/voxel_rcnn_feat_3_vehi.yaml --ckpt ../output/DA/nusc_kitti/source_only/voxel_rcnn_feat_3_vehi/default/ckpt/checkpoint_epoch_30.pth

However, as there is no DATA_CONFIG_TAR in cfgs/DA/nusc_kitti/source_only/voxel_rcnn_feat_3_vehi.yaml, this command will test the performance on the source dataset, instead of the target dataset, which is not my intension.

BOBrown commented 1 year ago

Hello: Thanks for approval and suggestion for our work. As you mentioned, we will revise the voxel_rcnn_feat_3_vehi.yaml in the next commit, and also, you could configure the DATA_CONFIG_TAR in the voxel_rcnn_feat_3_vehi.yaml, by referring to other cfg-files in nusc_kitti, such as https://github.com/PJLab-ADG/3DTrans/blob/master/tools/cfgs/DA/nusc_kitti/source_only/pv_rcnn_plus_sn_kitti.yaml

If you still have a problem, please contact us

YannnWanggg commented 1 year ago

Thank you for your reply!