search

WangYueFt / detr3d

MIT License
745 stars 140 forks source link

Question about Training Time #5

Closed zehuichen123 closed 2 years ago

zehuichen123 commented 2 years ago

Hi, Thx for your great work! I'm trying to rerun the training process of DETR3D on a 8-V100 machine using the default setting of detr3d_res101_gridmask_cbgs.py and the estimation of the training time is 8 days (I notice that the training epoch in paper is 12 while the code is 24, so for you the time should be 36h in this config?), which is 5.4x longer than the time you reported in the paper. Is this normal? I think 3090 should not be 5.4x faster than V100...

a1600012888 commented 2 years ago

Hi, zehuichen In the paper, we list the training time for 12 epoch with 8x 3090 without using cbgs. I am sorry we didn't make it clear.

CBGS dataset will over-sample the dataset, leading to around 3x-4x(I may remember the wrong number, but this should be close) more samples per epoch.

zehuichen123 commented 2 years ago

Thanks for your clarification.