Reproduce issue on Nuscenes dataset

[wzmsltw]

Dear authors, thank you for your work.

I tried to train Transfusion-L on nuscenes dataset. Since I encounter some issue about db dataset generate, I first train a model without db_sampler, and get the performance as: mAP: 60.6 NDS: 66.3 which is around 5 points lower than performance in paper.

After fix db problem, I retrain the model with db sampler but get even worse performance. Is this normal? Or should I finetune model without db sampler after training 20 epochs?

Thanks!

[XuyangBai]

Hi, for TransFusion-L I adopt the "fading strategy" which adopts the db_sampler but disenables it during the last 5 epochs. This strategy can reduce many false positives because the db_sample might put lots of objects at un-realistic positions and violates the real data distribution (but it is indeed useful to alleviate the long-tail problem so totally removing it will also bring a remarkable performance drop). So now you can just resume your training step (the one with db_sample) from 15 epochs and disenable the db_sample. And yes you can also fine-tune the model without db_sampler after 20 epochs but should take care the learning rate.

Best, Xuyang.

[wzmsltw]

Thanks for your quick reply! I will feedback the fine-tune results here later

[wzmsltw]

Hi Xuyang, After training 15 epoch with db and 5 epoch w/o db sampler, the TransFusion-L model achieve mAP 63.95 and NDS 68.93, which is higher than previous results but still a little lower than performance reported in paper. Is it normal? Since the pretrained model cannot be provided, can you provide the training log of the model with mAP 65.1 then I can check whether the loss is descended properly.

[XuyangBai]

Hi, I cannot share the official training log but I could share you with the training log from someone else's reproduction. I will contact you through email.

[wzmsltw]

Thanks so much!

[Zoeeeing]

@wzmsltw Hi, I would like to know the performance when you train the model with db sampler for 20 epochs?

[victor000000]

Hi, I cannot share the official training log but I could share you with the training log from someone else's reproduction. I will contact you through email.

I meet the reproduction problem too Could you please send me the training log you mentioned. yuanjun.li@jiduauto.com

[xjturjc]

Hi, I cannot share the official training log but I could share you with the training log from someone else's reproduction. I will contact you through email.

@XuyangBai Thanks for your work. I'm trying to train Transfusion-L on nuscenes dataset and I meet the reproduction problem too. Could you please send me the training log you mentioned. Thanks a lot. ren.jun.chen@163.com

[xtk8532704]

@XuyangBai Thanks for your work. I meet the reproduction problem too. Could you please send me the training log you mentioned. Thanks a lot. xian@ms.k.u-tokyo.ac.jp