search

dvlab-research / UVTR

Unifying Voxel-based Representation with Transformer for 3D Object Detection (NeurIPS 2022)
227 stars 18 forks source link

about knowledge transfer #18

Open Lotus-95 opened 2 years ago

Lotus-95 commented 2 years ago

Which config can perform the setting of Multi-mod to Camera?

Lotus-95 commented 2 years ago

There are only two files in /UVTR/tree/main/projects/configs/uvtr/camera_based/knowledge_distill. l2c and l2cs3.

Alexanderisgod commented 2 years ago

In the ranking of the nuScenes list, the NDS of camera is not as good as that of BEVFormer. Does it mean that it is not feasible to transfer point cloud knowledge to images, or the method is wrong?

yanwei-li commented 2 years ago

Hi @Lotus-95 , for multi-mod setting, you can directly modify the config according to the multi-mod setting here and turn the config model.type to "UVTRKDM".

yanwei-li commented 2 years ago

Hi @Alexanderisgod , there are different frameworks with different training settings. You can apply the knowledge transfer to BEVFormer, and should also find the improvement. Moreover, there are several methods better than the BEVFormer in the leaderboard. But it doesn't mean BEVFormer is wrong.

Alexanderisgod commented 2 years ago

I'm just not sure whether the structural information uncovered by knowledge transfer in your paper can be learned in the Transformer structure in BEVFormer. Or Transformer can learn structural information that is not weaker than knowledge transfer

Ljx-l commented 1 year ago

Hi @Lotus-95 , for multi-mod setting, you can directly modify the config according to the multi-mod setting here and turn the config model.type to "UVTRKDM".

If I turn the config model.type to "UVTRKDM", how to change the config about teacher*, it seems default to None?