search

waymo-research / waymo-open-dataset

Waymo Open Dataset
https://www.waymo.com/open
Other
2.68k stars 610 forks source link

range_image_top_pose do what? #51

Open AndyYuan96 opened 4 years ago

AndyYuan96 commented 4 years ago

Hi, I look at the source code of convert_range_image_to_point_cloud, and I find for TOP lidar point cloud , if I want to convert the point cloud to vehicle frame, the source code will first convert the point cloud according the TOP lidar's extrinsic to vehicle frame, and then use range_image_top_pose convert to another coordiante, and then use world_to_vehicle convert the point cloud to vehicle frame, why do that, the TOP lidar's extrinsic seems not enough? but for other lidars, just use the lidar's extrinsic is enough?

peisun1115 commented 4 years ago

range_image_top_pose specifies the vehicle (self-driving-car) pose for each range image pixel (point).

https://github.com/waymo-research/waymo-open-dataset/blob/master/waymo_open_dataset/dataset.proto#L155-L167

We only use raw range image for TOP lidar though we do plan to do similar stuff for other lidars in the future (no ETA).

xmyqsh commented 4 years ago

@peisun1115 I think range_image_top_pose here should be helpful for fixing the rolling shutter problem caused by the fast movement of lidar who has been referred https://github.com/waymo-research/waymo-open-dataset/issues/14#issuecomment-526900660

peisun1115 commented 4 years ago

What do you exactly mean by 'fixing'? What is the desired output after the 'fix'?

xmyqsh commented 4 years ago

@peisun1115 You are right. The raw range image with a non-empty range_image_pose_compressed exposes the rolling shutter problem. The rolling shutter problem is cause by fast-moving lidar here. In other word, the frequency of lidar is slower relatively when the velocity of the lidar is large. So we can fix it ourselves by interpolating the pose between lidar sweeps to animate a higher frequency lidar.

Phoebe-ovo commented 4 years ago

@xmyqsh I want to know how do you fix the rolling shutter problem? What do you mean by interpolation?

Manojbhat09 commented 4 years ago

@xmyqsh @peisun1115 Is there a code to do this fix? I am actually also concerned about this in actual deployment in self-driving vehicles. Input data processing could actually take a lot of processing power. Anyways, would be great if you could open-source the code for research into this.