search

TRAILab / CaDDN

Categorical Depth Distribution Network for Monocular 3D Object Detection (CVPR 2021 Oral)
Apache License 2.0
366 stars 62 forks source link

Can I train the Waymo model on OpenPCDet? #80

Closed Cc-Hy closed 3 years ago

Cc-Hy commented 3 years ago

HELLO, I'm trying to use the OpenPCDet to train the Waymo model. But it seems that it can not be directly applied, do you implement the Waymo model using OpenPCDet? If not, could you please give me some advice on what to do if I'd like to train the Waymo model with OpenPCDet. Thank you.

codyreading commented 3 years ago

My original Waymo Implementation used a KITTI formatted version of the dataset, which was before Waymo dataset support was added to OpenPCDet.

To use Waymo, you would need to add support for Waymo images on the data loader, and generate 2D labels of objects by projectintg the 3D boxes into the image. The point cloud range should be changed to POINT_CLOUD_RANGE: [2, -25.6, -4.0, 55.76, 25.6, 4.0]. Additionally, the Waymo images were down-sampled to meet memory constraints. Lastly, you will have to filter out all bounding boxes that are not in the front camera's FOV as CaDDN was only ran on the front camera.

Cc-Hy commented 3 years ago

Thanks, I noticed after the format transformation in OpenPCDet, there seems to be NO 'calib' file which is required in VFE and FOV. Another question is about the down-sampling, are there any requirements for the down-sampling resolution? e.g. has to be a multiple of 4 or 16, or has to be related to the point cloud range and the voxel size. Thank you.

codyreading commented 3 years ago

Yup, so you will also need to add support for the Waymo calibration information between image and LiDAR. In terms of downsampling, there are no requirements, but I recommend keeping the highest possible resolution that will fit in GPU memory.

Cc-Hy commented 3 years ago

Thanks a lot.

Cc-Hy commented 3 years ago

By the way, I notice that some methods use KITTI-depth(KITTI raw data) to do depth training, have you tried this? If not, do you think this would be helpful?

codyreading commented 3 years ago

Didn't try this, as there is an overlap betweeen KITTI-depth train and KITTI-OD validation, leading to biased results on the validation set (See here). This however would be helpful in the test set.

This would also be helpful if the novel split proposed here was used, but this paper was published after mine so never got around to trying it.