search

vzyrianov / lidargen

Official implementation of "Learning to Generate Realistic LiDAR Point Clouds" (ECCV 2022)
https://www.zyrianov.org/lidargen/
MIT License
122 stars 14 forks source link

Image to point cloud #5

Open zisismp4 opened 1 year ago

zisismp4 commented 1 year ago

Hello and thanks for sharing this great work! I am quite new with the LiDAR technology so please excuse my possibly naive question. After sampling, the output is 2-channel images. How can I recreate the pointcloudfrom them?

vzyrianov commented 1 year ago

Hello Zisis, thanks for checking out our work!

Each pixel value in a range image encodes a distance value. The {x,y} coordinate of that pixel contains information about the corresponding LiDAR point's {yaw, pitch} value. Given N x {depth, yaw, pitch} it is possible to project the points into 3D point cloud Nx3.

You can use this function in the codebase to do this transformation (transposing the input may be required) https://github.com/vzyrianov/lidargen/blob/4c3226cd7a53c241320923f936a434fdda3bad59/metrics/histogram/histogram.py#LL20C3-L20C3

zisismp4 commented 1 year ago

Thanks @vzyrianov for the quick response! I am a little bit confused! Let me describe what I am doing:

  1. I downloaded the pretrained model and the 3D LiDAR readings (just on folder because of memory issue)
  2. I ran python lidargen.py --sample --exp kitti_pretrained --config kitti.yml and two things were generated: image_grid.png and samples.pth in kitti_pretrained/image_samples/images/ and 0-7.pth in kitti_pretrained/unconditional_samples

Questions:

Exception has occurred: ValueError operands could not be broadcast together with shapes (65536,) (131072,) File "/home/zisis/Documents/lidargen/metrics/histogram/histogram.py", line 44, in range_image_to_point_cloud_fast pts[:, 0] = np.cos(yaw) np.cos(pitch) depth File "/home/zisis/Documents/lidargen/samples_load.py", line 34, in pcd = ri_2_pcd(un_samples_pth.numpy()) ValueError: operands could not be broadcast together with shapes (65536,) (131072,)

vzyrianov commented 1 year ago