search

microsoft / AirSim

Open source simulator for autonomous vehicles built on Unreal Engine / Unity, from Microsoft AI & Research
https://microsoft.github.io/AirSim/
Other
16.12k stars 4.5k forks source link

How do I get the segmentation ground truth of point clouds? #4104

Open catproof opened 2 years ago

catproof commented 2 years ago

I am trying to get a 3d point cloud of an environment the drone is flying in, which may be given by a lidar sensor. Is there a way to record what mesh any given point in the point cloud is colliding with/representing? For example, I have some point with x,y,z coordinates recorded at time t, and I want to know which mesh in the environment that point is on (ex: was the point given by the lidar shining on box_2, car_4, chair_0, or person_5, etc...)

This would be great for training and evaluating 3d segmentation algorithms.

rajat2004 commented 2 years ago

So you require more data than the segmentation field of the lidar data? FWIU, it contains the segmentation value of the mesh corresponding to each of the lidar data point, the same used in the segmentation images.

catproof commented 2 years ago

oh! I didn't know that was even a thing. I see it now in the documentation!

catproof commented 2 years ago

https://microsoft.github.io/AirSim/lidar/ "Segmentation: The segmentation of each lidar point's collided object"

I have yet to play around with this functionality. I am hoping it is straightforward to use. I suspect it indicates the name of the mesh each point collides with.