search

BichenWuUCB / SqueezeSeg

Implementation of SqueezeSeg, convolutional neural networks for LiDAR point clout segmentation
BSD 2-Clause "Simplified" License
561 stars 239 forks source link

Spherical projection #8

Open naranjuelo opened 6 years ago

naranjuelo commented 6 years ago

Hi, thanks for sharing your code! Are you planning to upload also the spherical projection part (described in your paper), in order to convert data from point cloud to projection images?

stratomaster31 commented 6 years ago

There is a nice quasi-equal implementation in the VoxelNet repository. The difference is that you have to convert the point cloud to spherical coordinates and work only with the azimuth and elevation. It may look tricky at first, but you'll for sure get it

naranjuelo commented 6 years ago

You mean this repository https://github.com/jeasinema/VoxelNet-tensorflow? I'm a little confused because if I understood it well, VoxelNet works with raw point cloud as input (which are then divided into voxels and some other processes), but they don't use any spherical projection or similar

stratomaster31 commented 6 years ago

Yes, this is the repository. The preprocessing routine is in "utils>preprocess.py". Although the voxelization is a 3D quantization of the point cloud, if you perform the same process with the raw point cloud in spherical coordinates but not counting the "r" variable, you will get the 2D quantized point cloud. The trick is convert every (x,y,z) in (theta,phi,r) and creating the 2D boundingbox imposing the constraints on theta and phi only.

naranjuelo commented 6 years ago

Okay, thanks! I will try that

agn-7 commented 6 years ago

@naranjuelo could you try this (create spherical projection with VoxelNet package)?

naranjuelo commented 6 years ago

I used my own implementation, but didn't get good results when I applied the pretrained model, so I guess there were some differences in the projection step

vvaquero commented 6 years ago

try to project one of the scans from which they provide their projection, so you will be able to check your differences. I guess they also normalize somehow the data ;)

tyunist commented 5 years ago

You can refer to @Durant35 solution: https://github.com/Durant35/SqueezeSeg/blob/master/src/nodes/segment_node.py #

stratomaster31 commented 5 years ago

Well, phi and theta are only used to transform Cartesian coordinates to spherical. The features of the points are then chosen as the Cartesian (x,y,z) and the intensity. For each point you can decide which feature to use, there is no a rule of thumb in that sense... Nevertheless, for these grids, phi and theta information are somehow embedded in the grids themselves, as they correspond to the index of each pixel. The phi and theta values will not be constant, but will have really small variance and they don't depend on the shape of the objects...

tyunist commented 5 years ago

Thanks @stratomaster31 for your prompt response. It's interesting to know that "The phi and theta values will not be constant, but will have really small variance and they don't depend on the shape of the objects...".

Is it the reason to choose spherical coordinate instead of other types to project 3D point cloud onto? And I I am trying to reproduce the result but got very limited evaluation result after 93000 steps. Here is what I done:

  1. Use training set provided
  2. Set load_pretrained_model path to file .pkl given
  3. Started training for 93000 steps.

But the current evaluation IoU is only 0.263.

stratomaster31 commented 5 years ago

Regarding to the projections, take a look to this pictures (they'll be more revealing than my previous reply)

x_surf azimuth_surf elevation_surf intensity_surf

As I mentioned, the elevation and azimuth features will depend only (almost only) on the pixel index for those points in which the LiDAR has hit an object. So, in the end, with phi and theta you will be more likely to encode a mask indicating where in the projection valid LiDAR points are present. The surf plots for phi and theta will not be very different from one cloud to another, except for this masking... The spherical projection is chosen due to the nature of the LiDAR sensor, which is rotating in the azimuth coordinate, and sends laser beams in the elevation one. Nevertheless, if you change the projection you'll use anoother coordinate system, for instance, for a BeV projection it is a better idea to work on the Cartesian system (x,y) for making the height map. Another interesting aspect is to asses whether including x,y and z is worth or not, since the majority of the information relies on I and R.

About the training... have you cloned the repository or have you coded the net by yourself?

omartriki1 commented 5 years ago

Hi, Can anyone tell me how to implement the spherical projection? I want to use lidar 3D point cloud but I have to project them before. Can anyone share his implementation?

lawrencekiba commented 5 years ago

Hi, Can anyone tell me how to implement the spherical projection? I want to use lidar 3D point cloud but I have to project them before. Can anyone share his implementation?

Please refer the comment above by @tyunist

https://github.com/BichenWuUCB/SqueezeSeg/issues/8#issuecomment-426916091

ghost commented 4 years ago

Hello good people, I am also looking for the implementation of the spherical projection. I have converted the x,y,z co ordinate into pitch and yaw and then I got the spherical projection u and v. but i am not getting any image. Any help would be appreciated. Thank you.