mpelchat04
commented
4 years ago This ticket is a duplicate of #2
Closed ymoisan closed 4 years ago
mpelchat04
commented
4 years ago This ticket is a duplicate of #2
Lidar point cloud classification
Context
Airborne lidar systems generate data in the form of point clouds. The American Society for Photogrammetry and Remote Sensing (ASPRS) -- which now tags itself as The Imaging and Geospatial Information Society -- has a Lidar division the mission of which is "to provide a forum for collection development and dissemination of information related to the best practices in developing maintaining and operating kinematic laser scanners and associated sensors".
The LAS Working Group maintains the LAS file format standard, which is the default format for storing lidar data points. The latest version of the standard, LAS 1.4, provides the structure of an individual point data record:
Table 1: Point Data Record Format 0
The classification item in the table above is further specified in the following two tables:
Table 2: Classification Bit definition (field encoding)
Table 3: ASPRS Standard LIDAR Point Class Values
Therefore, the format for the classification field is a bit encoding with the lower five bits used for the class value (as shown in table 3 above) and the three high bits used for flags.
Problem
The issue here is that the classification of lidar points is both expensive (typically 30 % of total data acquisition cost) and rather unreliable. We would like to devise a deep learning approach to help us classify data points independently, that is irrespective of class values assigned (or not) by the data provider.
Literature review
There are roughly two types of methods for lidar point cloud classification using DL:
The methods we are most interested in are those that have a PyTorch implementation.
CNN
Classifying airborne LiDAR point clouds via deep features learned by a multi-scale convolutional neural network
"With several selected attributes of LiDAR point clouds, our method first creates a group of multi-scale contextual images for each point in the data using interpolation. Taking the contextual images as inputs, a multi-scale convolutional neural network (MCNN) is then designed and trained to learn the deep features of LiDAR points across various scales. A softmax regression classifier (SRC) is finally employed to generate classification results of the data with a combination of the deep features learned from various scales. "
3D Point Cloud Classification and Segmentation using 3D Modified Fisher Vector Representation for Convolutional Neural Networks
"The point cloud ... the common solution [for classification] of transforming the data into a 3D voxel grid introduces its own challenges, mainly large memory size ... we propose a novel 3D point cloud representation called 3D Modified Fisher Vectors (3DmFV) ... it combines the discrete structure of a grid with continuous generalization of Fisher vectors ... Using the grid enables us to design a new CNN architecture for point cloud classification and part segmentation."
Large-scale Point Cloud Semantic Segmentation with Superpoint Graphs
Point clouds
Using CNNs implies rasterization of point attribute values, which in turn implies degradation of the level of information contained in the point cloud. Ideally, we would like our algorithms to work directly in the point cloud. The main problem here is the data volume. We should strive to find approaches that minimize data I/O. Things like Entwine may allow us to request point data for training or inference via web services and therefore avoid data duplication.
PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation
"This, however, renders data unnecessarily voluminous and causes issues. In this paper, we design a novel type of neural network that directly consumes point clouds and well respects the permutation invariance of points in the input. Our network, named PointNet, provides a unified architecture for applications ranging from object classification, part segmentation, to scene semantic parsing. Though simple, PointNet is highly efficient and effective."
Spherical Convolutional Neural Network for 3D Point Clouds
"We propose a neural network for 3D point cloud processing that exploits `spherical' convolution kernels and octree partitioning of space. The proposed metric-based spherical kernels systematically quantize point neighborhoods to identify local geometric structures in data ... The network architecture itself is guided by octree data structuring that takes full advantage of the sparse nature of irregular point clouds. We specify spherical kernels with the help of neurons in each layer that in turn are associated with spatial locations. We exploit this association to avert dynamic kernel generation during network training, that enables efficient learning with high resolution point clouds. We demonstrate the utility of the spherical convolutional neural network for 3D object classification on standard benchmark datasets."
Interactive Visualization of 10M+ 3D Points with New Open-Source Python Package PPTK
"The PPTK viewer is part of a larger effort of developing a Python toolkit for not only visualizing but also processing point data."