search

Cc-Hy / CMKD

Cross-Modality Knowledge Distillation Network for Monocular 3D Object Detection (ECCV 2022 Oral)
Apache License 2.0
107 stars 9 forks source link

Huge fluctuations in eval results with test.py #35

Closed charmeleonz closed 1 year ago

charmeleonz commented 1 year ago

Hi, I ran into some very strange results. I pre-trained a SECOND teacher for 80 epochs.

When evaluating the 60th and 80th epochs' checkpoints with test.py, the results are good (Car-Mod>80), but the 70th epoch gives extremely bad results (Car-Mod~10, average predicted number of objects: 300+).

I also followed the advice in another issue to modify init.py under pcdet/datasets/ but it didn't help.

The SECOND model is trained on Train and evaluated on Val.

Screenshot from 2023-04-15 09-39-37

Cc-Hy commented 1 year ago

Hi, how are the losses going at the abnormal epoch? And how is the performance just before or after this epoch (i.e., 69 and 71)? Maybe you can check these out for some leads.