Deep learning applied to lidar point cloud classification

[ymoisan]

Context

Classification of lidar point clouds is resource-intensive as it generally involves human interaction to obtain decent results. Furthermore, it has been shown that classification results and accuracies vary between data producers. Finally, classification amounts to about 30% of lidar data acquisition costs.

Given that context, how can Deep Learning (DL) help in the classification process ?

Issues

DL architectures used in imagery are mostly CNNs. Convolution implies pixels. However, lidar cloud points do not correspond to pixels and therefore CNNs are likely not applicable in this case.

Thoughts

Could we use prior data (DEM, imagery, etc.) to initialize weights in a scene-specific rough manner rather than using a random initialization ?

Could we use standard unsupervised clustering algorithms like K-Means or DBSCAN ? If so, what is the minimum number of points we can run clustering algorithms against ?

References & projects

Classifying airborne LiDAR point clouds via deep features learned by a multi-scale convolutional neural network Paywalled, unfortunately ...

Deep Semantic Classification for 3D LiDAR Data. Key to using CNNs with point clouds: "The input to our network is a set of three channel 2D images generated by unwrapping 360^^0 3D LiDAR data onto a spherical 2D plane".

PyLidar; based on not well-known libraries ([SPDLibb[(http://www.spdlib.org/)), but developed by a government organization (New South Wales, Australia) and still looks pretty interesting: "Supported formats are: SPD V3, SPD V4, Riegl RXP, LAS, LVIS, ASCII and Pulsewaves"

Deep learning-based tree classification using mobile LiDAR data

Deep Semantic Classification for 3D LiDAR Data. Key to using CNNs with point clouds: "The input to our network is a set of three channel 2D images generated by unwrapping 360^^0 3D LiDAR data onto a spherical 2D plane"

PointNet : "Deep Learning on Point Sets for 3D Classification and Segmentation" and it's based on this work from stanford University.

[ali-robot]

Hello,

What kind of LIDAR do you use? Is your deep learning work applicable to data issued from a velodyne or velodyne-like LIDAR on the roof of a car ?

[mpelchat04]

Hi @ali-robot,
The goal is to be able to apply DL to any pointcloud, regardless of the sensor. At this point, we only started with a literature review to figure the best way to add this feature to Geo-deep-learning. That being said, our main concern is to apply this to aerial LiDAR, mainly for cartographic purposes.

[ali-robot]

Hi @mpelchat04 I am at a similar stage, i.e. literature review, and the purpose is cartography (SLAM) as well but using lidars mounted on vehicles (similar to velodyne lidars but not limited to velodyne of course) instead of aerial lidars.

[mpelchat04]

Some other infos were in a duplicate ticket: #69

[valhassan]

Independent project, not related to GDL.