Intensity scaling in FSDv2

tusen-ai / SST

Code for a series of work in LiDAR perception, including SST (CVPR 22), FSD (NeurIPS 22), FSD++ (TPAMI 23), FSDv2, and CTRL (ICCV 23, oral).

Apache License 2.0

801 stars 102 forks source link

Intensity scaling in FSDv2 #150

Open ArseniuML opened 1 year ago

ArseniuML commented 1 year ago

I want to test FSDv2 with downloaded Waymo weights on my point clouds. However, my LIDAR outputs intensity as UINT8 (0-255), and in Waymo data intensity maximum in various frames can be as large as ~33000 or ~80000. If I feed my data to FSDv2 as is, it produces garbage as prediction results... How do I have to transform intensity from my datasets?

ArseniuML commented 1 year ago

The same question about feeding my data to FSDv2 with Argo2 weights.

Abyssaledge commented 1 year ago

We normalize intensity here: https://github.com/tusen-ai/SST/blob/main/mmdet3d/models/detectors/single_stage_fsd.py#L287 Sorry for the hard coding. I suggest training a model without intensity, only taking xyz as input. Otherwise, even if you normalize the intensity to a similar range, the intensity distribution of WOD and your data may still be different.

ArseniuML commented 2 months ago

I don't think there's much point in applying tanh to intensity (especially non-normalized), because we end with degenerated distributions.