search

open-mmlab / mmdetection3d

OpenMMLab's next-generation platform for general 3D object detection.
https://mmdetection3d.readthedocs.io/en/latest/
Apache License 2.0
5.18k stars 1.52k forks source link

PointPillars' training logs confusion #1463

Open TuanTNG opened 2 years ago

TuanTNG commented 2 years ago

Hi MMDetection3D teams,

I am re-producing PointPillars config, this config. However, when I read its training logs, this logs, I found something wrong.

The total number of training samples of KITTI is 3712 as in this readme. When training PointPillars configs with repeating KITTI dataset by 2 times, samples_per_gpu=6 and 8 GPUs. It must result in around 155 (~=3712*2/8/6) iterations per one epoch. However, the training log shows 1200 iterations per one epoch.

Is there anything wrong?

I found that other config and its log which trained using 4 GPUs & samples_per_gpu=4, are correct (the log shows 450 (~= 3712*2/4/4) iterations per epoch).

Thank you so much.

Xiangxu-0103 commented 2 years ago

Yeah, I found the training logs for pp_kitti-3class and pp_kitti-car are little different from the corresponding configs.

Xiangxu-0103 commented 2 years ago

The potential difference has been fixed in #1486. I found the training log for pp_3class is exactly trained with only one GPU, which causes the iteration 1200 per epoch.

Tai-Wang commented 2 years ago

@ZCMax Please have a look at this issue and update the log files if necessary.