about augment_and_scale_3d

[iris0329]

Hi, Thank you for opening up such a great job! In xmuda/data/semantic_kitti/semantic_kitti_dataloader.py I found there is a function named augment_and_scale_3d. The question is that if only the points is changed but the image is not changed, wouldn't the image and points not correspond?

[maxjaritz]

Hi, this is a good question. There are 2 types of 2D-3D augmentations: 1) independent augmentations that do not change 2D-3D correspondence and 2) dependent augmentations that change 2D-3D correspondence. The 2D-3D correspondence is defined by by projecting the 3D points into the 2D image. In the dataloader, I call them image points points_img when they are float values or image indices img_indices when they are integer values. When the order of points does not change, the augmentations are actually independent.

1) Examples for independent augmentations

• 2D Color jitter: the projection is still valid as only the color values change
• 3D rotation, axis flipping, rotation: the order of 3D points still corresponds to the order of img_indices. The semantic prediction will still mean the same as the corresponding location in the image.

2) Examples for dependent augmentations

• image cropping: the 3D points outside of the cropped image region need to be discarded
• left-right image flipping: the img_indices need to be flipped as well

Hope it helps!

[iris0329]

@maxjaritz Thank you very much for your detailed answer, I understand now. Best, Iris