search

weiyithu / LiDAR-Distillation

[ECCV 2022] LiDAR Distillation: Bridging the Beam-Induced Domain Gap for 3D Object Detection
Apache License 2.0
105 stars 10 forks source link

Performance #2

Open CBY-9527 opened 2 years ago

CBY-9527 commented 2 years ago

Hello, I have a question. I sampled waymo 1/20 for training, and directly tested nuscenes. The performance (26.44/12.11) is similar to that given in Table 1 of the paper. However, the distillation performance is 27.37/13.46, which is quite different from that given in the paper (40.23/19.12). The performance of "our" in Table 1 is "distillation +sn" or only "distillation".

weiyithu commented 2 years ago

It is "distillation". Have you follow https://github.com/weiyithu/LiDAR-Distillation/blob/main/docs/GETTING_STARTED.md to train the model?

CBY-9527 commented 2 years ago

It is "distillation". Have you follow https://github.com/weiyithu/LiDAR-Distillation/blob/main/docs/GETTING_STARTED.md to train the model?

First, I trained the PooointPillars for waymo 64-beam, and found the best model. Then, the best model is distilled to waymo 32-beam, and used the distilled model to test nuscenes. The above steps seem be consistent with those in readme. The only difference is that my waymo is 1/20 sampled, but the performance of training sampled waymo and directly testing nuscenes is similar to that given in the table 1.

weiyithu commented 2 years ago

As metioned in our paper, the beams of nuscenes is equal to 16^ beam but not 32 beam. However, I think this is not the reason because 32-beam model should be better than 64-beam model (Table 8). I think there must be a bug. If you are Chinese, here is my wechat 18600500891 and we can discuss in wechat.