awesome-point-cloud-registration

A curated list of resources on point cloud registration inspired by awesome-computer-vision. Work-in-progress. All contributions are welcome and appreciated.

This list focuses on the rigid registration between point clouds.

Table of Contents

• Coarse Registration (Global Registration)
  • Feature Matching Based
    • Keypoint Detection
    • Feature Description
    • Outlier Rejection
    • Graph Algorithms
  • End-to-End
  • Randomized
  • Probablistic
• Fine Registration (Local Registration)
  • Learning-based
  • Traditional
• Datasets
• Tools

Coarse Registration

The coarse registration methods (or global registration) aligns two point clouds without an initial guess. We broadly classified these methods into feature matching based, end-to-end, randomized and probabilistic. Most of the learning based methods are focusing on some specific step in the feature matching based algorithms.

Feature Matching Based

The feature-matching based registration algorithms generally follow a two-stage workflow: determining correspondence and estimate the transformation. The correspondence establishing stage usually follow the four-step pipeline: keypoint detection, feature description, descriptor matching and outlier rejection. The nearest neighbor matching is the de-facto matching strategy, but could be replaced by learnable matching stategies. We also include some papers which adopt the graph algorithms for the matching and outlier rejection problem.

Keypoint Detection

• HKS: A Concise and Provably Informative Multi‐Scale Signature Based on Heat Diffusion. CGF'2009 [paper]
• Harris3D: a robust extension of the harris operator for interest point detection on 3D meshes. VC'2011 [paper]
• Intrinsic shape signatures: A shape descriptor for 3D object recognition. ICCV'2009 [paper]
• Learning a Descriptor-Specific 3D Keypoint Detector. ICCV'2015 [paper]
• 3DFeat-Net: Weakly Supervised Local 3D Features for Point Cloud Registration. ECCV'2018 [paper] [code]
• USIP: Unsupervised Stable Interest Point Detection from 3D Point Clouds. ICCV'2019 [paper] [code]
• D3Feat: Joint Learning of Dense Detection and Description of 3D Local Features. CVPR'2020 [paper] [code]
• PointCloud Saliency Maps. ICCV'2019 [paper] [code]
• SK-Net: Deep Learning on Point Cloud via End-to-end Discovery of Spatial Keypoints. AAAI'2020 [paper]
• SKD: Unsupervised Keypoint Detecting for Point Clouds using Embedded Saliency Estimation. arxiv'2019 [paper]
• Fuzzy Logic and Histogram of Normal Orientation-based 3D Keypoint Detection For Point Clouds. PRL'2020 [paper]
• MaskNet: A Fully-Convolutional Network to Estimate Inlier Points. 3DV'2020 [paper] [code]
• PREDATOR: Registration of 3D Point Clouds with Low Overlap. arxiv'2020 [paper] [code]
• SC3K: Self-supervised and Coherent 3D Keypoints Estimation from Rotated, Noisy, and Decimated Point Cloud Data. ICCV'2023 [paper] [code]

Survey:

• Performance Evaluation of 3D Keypoint Detectors. IJCV'2013 [paper]

Feature Description

• Spin Image: Using spin images for efficient object recognition in cluttered 3D scenes. TPAMI'1999 [paper]
• USC: Unique shape context for 3D data description. 3DOR'2010 [paper]
• 3DShapeContext: Recognizing Objects in Range Data Using Regional Point Descriptors. ECCV'2004 [paper]
• SHOT: Unique Signatures of Histograms for Local Surface Description. ECCV'2010 [paper]
• FPFH: Fast Point Feature Histograms (FPFH) for 3D registration. ICRA'2009 [paper]
• RoPS: 3D Free Form Object Recognition using Rotational Projection Statistics. WACV'2013 [paper]
• CGF: Learning Compact Geometric Features. ICCV'2017 [paper]
• 3DMatch: Learning Local Geometric Descriptors from RGB-D Reconstructions. CVPR'2017 [paper] [code]
• End-to-end learning of keypoint detector and descriptor for pose invariant 3D matching. CVPR'2018 [paper]
• PPFNet: Global Context Aware Local Features for Robust 3D Point Matching. CVPR'2018 [paper]
• 3DFeat-Net: Weakly Supervised Local 3D Features for Point Cloud Registration. ECCV'2018 [paper] [code]
• MVDesc: Learning and Matching Multi-View Descriptors for Registration of Point Clouds. ECCV'2018 [paper] [code]
• FoldingNet: Point Cloud Auto-encoder via Deep Grid Deformation. CVPR'2018 [paper] [code]
• PPF-FoldNet: Unsupervised Learning of Rotation Invariant 3D Local Descriptors. ECCV'2018 [paper] [code]
• 3D Local Features for Direct Pairwise Registration. CVPR'2019 [paper]
• 3D Point-Capsule Networks. CVPR'2019 [paper] [code]
• The Perfect Match: 3D Point Cloud Matching with Smoothed Densities. CVPR'2019 [paper] [code]
• FCGF: Fully Convolutional Geometric Features. ICCV'2019 [paper] [code]
• Learning an Effective Equivariant 3D Descriptor Without Supervision. ICCV'2019 [paper]
• D3Feat: Joint Learning of Dense Detection and Description of 3D Local Features. CVPR'2020 [paper] [code]
• End-to-End Learning Local Multi-view Descriptors for 3D Point Clouds. CVPR'2020 [paper] [code]
• LRF-Net- Learning Local Reference Frames for 3D Local Shape Description and Matching. arxiv'2020 [paper]
• DH3D: Deep Hierarchical 3D Descriptors for Robust Large-Scale 6DoF Relocalization. ECCV'2020 [paper] [code]
• Distinctive 3D local deep descriptors. arxiv'2020 [paper] [code]
• SpinNet: Learning a General Surface Descriptor for 3D Point Cloud Registration. CVPR'2021 [paper] [code]
• PREDATOR: Registration of 3D Point Clouds with Low Overlap. CVPR'2021 [paper] [code]
• Self-supervised Geometric Perception. CVPR'2021 [paper] [code]
• 3D Point Cloud Registration with Multi-Scale Architecture and Self-supervised Fine-tuning. arxiv'2021 [paper] [code]
• Generalisable and Distinctive (GeDi) 3D local deep descriptors for point cloud registration. arxiv'2021 [paper] [code]
• Neighborhood Normalization for Robust Geometric Feature Learning. CVPR'2021 [paper] [code]
• UnsupervisedR&R: Unsupervised Point Cloud Registration via Differentiable Rendering. CVPR'2021 [paper] [code]
• Bootstrap Your Own Correspondences. ICCV'2021 [paper] [code]
• WSDesc: Weakly Supervised 3D Local Descriptor Learning for Point Cloud Registration. TVCG'2022 [paper] [code]
• You Only Hypothesize Once: Point Cloud Registration with Rotation-equivariant Descriptors. ICCV'2021 [paper] [code]
• P2-Net: Joint Description and Detection of Local Features for Pixel and Point Matching. ICCV'2021 [paper]
• Distinctiveness oriented Positional Equilibrium for Point Cloud Registration. ICCV'2021 [paper]
• CoFiNet: Reliable Coarse-to-fine Correspondences for Robust Point Cloud Registration. NeurIPS'2021 [paper] [code]
• IMFNet: Interpretable Multimodal Fusion for Point Cloud Registration. arxiv'2021 [paper]
• Lepard: Learning partial point cloud matching in rigid and deformable scenes. CVPR'2022 [paper] [code]
• Fast and Robust Registration of Partially Overlapping Point Clouds. RA-L'2021 [paper] [code]
• Geometric Transformer for Fast and Robust Point Cloud Registration. CVPR'2022 [paper] [code]
• ImLoveNet: Misaligned Image-supported Registration Network for Low-overlap Point Cloud Pairs. SIGGRAPH'2022 [paper]
• Learning to Register Unbalanced Point Pairs. arxiv'2022 [paper]

Survey:

• A Comprehensive Performance Evaluation of 3D Local Feature Descriptors. IJCV'2015 [paper]
• Evaluating Local Geometric Feature Representations for 3D Rigid Data Matching. ICIP'2019 [paper]

Outlier Rejection

We also include the algorithms designed for finding matching between keypoints given descriptors (which replaces nearest-neighbor-searching) in this section.

• RANSAC: Random Sample Consensus: A Paradigm for Model Fitting with Applications to Image Analysis and Automated Cartography. 1981 [paper]
• Locally Optimized RANSAC. 2003 [paper]
• Graph-cut RANSAC. CVPR'2018 [paper] [code]
• MAGSAC: Marginalizing Sample Consensus. CVPR'2019 [paper] [code]
• VFC: A Robust Method for Vector Field Learning with Application To Mismatch Removing. CVPR'2011 [paper]
• In Search of Inliers: 3D Correspondence by Local and Global Voting. CVPR'2014 [paper]
• FGR: Fast Global Registration. ECCV'2016 [paper] [code]
• Ranking 3D Feature Correspondences Via Consistency Voting. PRL'2019 [paper]
• An Accurate and Efficient Voting Scheme for a Maximally All-Inlier 3D Correspondence Set. TPAMI'2020 [paper]
• GORE: Guaranteed Outlier Removal for Point Cloud Registration with Correspondences. TPAMI'2018 [paper] [code]
• A Polynomial-time Solution for Robust Registration with Extreme Outlier Rates. RSS'2019 [paper]
• Graduated Non-Convexity for Robust Spatial Perception: From Non-Minimal Solvers to Global Outlier Rejection. ICRA'2020 [paper]
• TEASER: Fast and Certifiable Point Cloud Registration. T-RO'2020 [paper] [code]
• One Ring to Rule Them All: Certifiably Robust Geometric Perception with Outliers. NeurIPS'2020 [paper]
• SDRSAC: Semidefinite-Based Randomized Approach for Robust Point Cloud Registration without Correspondences. CVPR'2019 [paper] [code]
• Robust Low-Overlap 3D Point Cloud Registration for Outlier Rejection. ICRA'2019 [paper]
• ICOS: Efficient and Highly Robust Rotation Search and Point Cloud Registration with Correspondences. arxiv'2021 [paper]
• Fast Semantic-Assisted Outlier Removal for Large-scale Point Cloud Registration. arxiv'2022 [paper]
• A Single Correspondence Is Enough: Robust Global Registration to Avoid Degeneracy in Urban Environments. ICRA'2022 [paper] [code]
• SC^2-PCR: A Second Order Spatial Compatibility for Efficient and Robust Point Cloud Registration. CVPR'2022 [paper] [code]

Learning based (including 2D outlier rejection methods)

• Learning to Find Good Correspondences. CVPR'2018 [paper] [code]
• NM-Net: Mining Reliable Neighbors for Robust Feature Correspondences. CVPR'2019 [paper] [code]
• OANet: Learning Two-View Correspondences and Geometry using Order-Aware Network. ICCV'2019 [paper] [code]
• ACNe: Attentive Context Normalization for Robust Permutation-Equivariant Learning. CVPR'2020 [paper] [code]
• SuperGlue: Learning Feature Matching with Graph Neural Networks. CVPR'2020 [paper] [code]
• 3DRegNet: A Deep Neural Network for 3D Point Registration. CVPR'2020 [paper] [code]
• Deep Global Registration. CVPR'2020 [paper] [code]
• 3D Correspondence Grouping with Compatibility Features. arxiv'2020 [paper]
• PointDSC: Robust Point Cloud Registration using Deep Spatial Consistency. CVPR'2021 [paper] [code]
• StickyPillars: Robust and Efficient Feature Matching on Point Clouds using Graph Neural Networks. CVPR'2021 [paper]
• HRegNet: A Hierarchical Network for Large-scale Outdoor LiDAR Point Cloud Registration. ICCV'2021 [paper] [code]
• Keypoint Matching for Point Cloud Registration using Multiplex Dynamic Graph Attention Networks. RA-L'2021 [paper]
• Deep Hough Voting for Robust Global Registration. ICCV'2021 [paper]
• End-to-end Learning the Partial Permutation Matrix for Robust 3D Point Cloud Registration. arxiv'2021 [paper]
• TriVoC: Efficient Voting-based Consensus Maximization for Robust Point Cloud Registration with Extreme Outlier Ratios. arxiv'2021 [paper]
• COTReg: Coupled Optimal Transport based Point Cloud Registration. arxiv'2021 [paper]
• DetarNet: Decoupling Translation and Rotation by Siamese Network for Point Cloud Registration. AAAI'2022 [paper] [code]
• Multi-instance Point Cloud Registration by Efficient Correspondence Clustering. CVPR'2022 [paper] [code]

Survey

• A Performance Evaluation of Correspondence Grouping Methods for 3D Rigid Data Matching. TPAMI'2019 [paper]

Graph Algorithms

• A Graduated Assignment Algorithm for Graph Matching. TPAMI'1996 [paper]
• A Spectral Technique for Correspondence Problems Using Pairwise Constraints. ICCV'2005 [paper] [code]
• Balanced Graph Matching. NIPS'2006 [paper] [code]
• Feature Correspondence via Graph Matching: Models and Global Optimization. ECCV'2008 [paper]
• An Integer Projected Fixed Point Method for Graph Matching and MAP Inference. NIPS'2009 [paper] [code]
• Optimal Correspondences From Pairwise Constraints. ICCV'2009 [paper]
• Reweighted Random Walks for Graph Matching. ECCV'2010 [paper]
• Maximal Cliques Based Rigid Body Motion Segmentation with a RGB-D Camera. ACCV'2012 [paper]
• A Probabilistic Approach to Spectral Graph Matching. TPAMI'2013 [paper]
• A Practical Maximum Clique Algorithm for Matching with Pairwise Constraints. arxiv'2019 [paper]
• ROBIN: a Graph-Theoretic Approach to Reject Outliers in Robust Estimation using Invariants. arxiv'2020 [paper]
• CLIPPER A Graph-Theoretic Framework for Robust Data Association. arxiv'2020 [paper]
• PointDSC: Robust Point Cloud Registration using Deep Spatial Consistency CVPR'2021 [paper] [code]
• Pairwise Point Cloud Registration Using Graph Matching and Rotation-invariant Features. arxiv'2021 [paper]
• Pyramid Semantic Graph-based Global Point Cloud Registration with Low Overlap. IROS'2023 [paper] [code]
• G3Reg: Pyramid Graph-based Global Registration using Gaussian Ellipsoid Model. arxiv'2023 [paper] [code]

End-to-End

Some papers perform end-to-end registration by directly predicting a rigid transformation aligning two point clouds without explicitly following the detection -- description -- outlier filtering pipepline. While they works well on object-centric datasets, the performance on real-world scene registration is not satisfactory.

• PointNetLK: Robust & Efficient Point Cloud Registration using PointNet. CVPR'2019 [paper] [code]
• Deep Closest Point: Learning Representations for Point Cloud Registration. ICCV'2019 [paper] [code]
• PRNet: Self-Supervised Learning for Partial-to-Partial Registration. NeurIPS'2019 [paper] [code]
• AlignNet-3D: Fast Point Cloud Registration of Partially Observed Objects. 3DV'2019 [paper] [code]
• RPM-Net: Robust Point Matching using Learned Features. CVPR'2020 [paper] [code]
• Feature-metric Registration: A Fast Semi-supervised Approach for Robust Point Cloud Registration without Correspondences. CVPR'2020 [code] [code]
• Iterative Distance-Aware Similarity Matrix Convolution with Mutual-Supervised Point Elimination for Efficient Point Cloud Registration. ECCV'2020 [paper] [code]
• Self-supervised Point Set Local Descriptors for Point Cloud Registration. arxiv'2020 [paper]
• Learning 3D-3D Correspondences for One-shot Partial-to-partial Registration. arxiv'2020 [paper]
• Robust Point Cloud Registration Framework Based on Deep Graph Matching. CVPR'2021 [paper] [code]
• RPSRNet: End-to-End Trainable Rigid Point Set Registration Network using Barnes-Hut 2^D-Tree Representation. CVPR'2021 [paper]
• Deep Weighted Consensus (DWC) Dense correspondence confidence maps for 3D shape registration. arxiv'2021 [paper]
• OMNet: Learning Overlapping Mask for Partial-to-Partial Point Cloud Registration. arxiv'2021 [paper]
• FINet: Dual Branches Feature Interaction for Partial-to-Partial Point Cloud Registration. AAAI'2022 [paper] [code]
• PointNetLK Revisited. CVPR'2021 [paper]
• Point Cloud Registration using Representative Overlapping Points. arxiv'2021 [paper] [code]
• Correspondence-Free Point Cloud Registration with SO(3)-Equivariant Implicit Shape Representations. arxiv'2021 [paper]
• Geometry Guided Network for Point Cloud Registration. RA-L'2021 [paper]
• (Just) A Spoonful of Refinements Helps the Registration Error Go Down. ICCV'2021 [paper] [code]
• A Robust Loss for Point Cloud Registration. ICCV'2021 [paper]
• Sampling Network Guided Cross-Entropy Method for Unsupervised Point Cloud Registration. ICCV'2021 [paper] [code]
• DeepBBS: Deep Best Buddies for Point Cloud Registration. 3DV'2021 [paper] [code]
• PCAM: Product of Cross-Attention Matrices for Rigid Registration of Point Clouds. ICCV'2021 [paper] [code]
• Feature Interactive Representation for Point Cloud Registration. ICCV'2021 [paper]
• DeepPRO: Deep Partial Point Cloud Registration of Objects. ICCV'2021 [paper]
• What Stops Learning-based 3D Registration from Working in the Real World? arxiv'2021 [paper]
• Full Transformer Framework for Robust Point Cloud Registration with Deep Information Interaction. arxiv'2021 [paper] [code]
• Reliable Inlier Evaluation for Unsupervised Point Cloud Registration. AAAI'2022 [paper]
• VRNet: Learning the Rectified Virtual Corresponding Points for 3D Point Cloud Registration. arxiv'2022 [paper]
• REGTR: End-to-end Point Cloud Correspondences with Transformers. CVPR'2022 [paper] [code]
• UTOPIC: Uncertainty-aware Overlap Prediction Network for Partial Point Cloud Registration. CGF'2022 [paper] [code]

Randomized

• RANSAC: Random Sample Consensus: A Paradigm for Model Fitting with Applications to Image Analysis and Automated Cartography. 1981 [paper]
• 4PCS: 4-points Congruent Sets for Robust Pairwise Surface Registration. TOG'2008 [paper]
• Model Globally, Match Locally: Efficient and Robust 3D Object Recognition. CVPR'2010 [paper] [code]
• Super 4PCS: Fast Global Pointcloud Registration via Smart Indexing. CGF'2014 [paper] [code]

Probabilistic

• Point Set Registration: Coherent Point Drift. TPAMI'2010 [paper] [code]
• Robust Point Set Registration Using Gaussian Mixture Models. TPAMI'2011 [paper] [code]
• A Generative Model for the Joint Registration of Multiple Point Sets. ECCV'2014 [paper]
• Aligning the Dissimilar: A Probabilistic Method for Feature-based Point Set Registration. ICPR'2016 [paper]
• A Probabilistic Framework for Color-Based Point Set Registration. CVPR'2016 [paper]
• Density Adaptive Point Set Registration. CVPR'2018 [paper] [code]
• HGMR: Hierarchical Gaussian Mixtures for Adaptive 3D Registration. ECCV'2018 [paper]
• Robust Feature-Based Point Registration Using Directional Mixture Model. arxiv'2019 [paper]
• FilterReg: Robust and Efficient Probabilistic Point-Set Registration using Gaussian Filter and Twist Parameterization. CVPR'2019 [paper] [code]
• PointGMM: a Neural GMM Network for Point Clouds. CVPR'2020 [paper] [code]
• DeepGMR: Learning Latent Gaussian Mixture Models for Registration. ECCV'2020 [paper] [code]
• Registration Loss Learning for Deep Probabilistic Point Set Registration. 3DV'2020 [paper] [code]
• A Termination Criterion for Probabilistic PointClouds Registration. arxiv'2020 [paper]
• LSG-CPD: Coherent Point Drift with Local Surface Geometry for Point Cloud Registration. ICCV'2021 [paper]

Others

• Gravitational Approach for Point Set Registration. CVPR'2016 [paper]
• Accelerated Gravitational Point Set Alignment with Altered Physical Laws. ICCV'2019 [paper]
• Fast Gravitational Approach for Rigid Point Set Registration with Ordinary Differential Equations. arxiv'2020 [paper]
• Minimal Solvers for Mini-Loop Closures in 3D Multi-Scan Alignment. CVPR'2019 [paper]
• Minimal Solvers for 3D Scan Alignment With Pairs of Intersecting Lines. CVPR'2020 [paper]
• Learning multiview 3D point cloud registration. CVPR'2020 [paper] [code]
• A Dynamical Perspective on Point Cloud Registration. arxiv'2020 [paper]
• Plane Pair Matching for Efficient 3D View Registration. arxiv'2020 [paper]
• On Bundle Adjustment for Multiview Point Cloud Registration. RA-L'2021 [paper]
• Provably Approximated Point Cloud Registration. ICCV'2021 [paper]
• GenReg: Deep Generative Method for Fast Point Cloud Registration. arxiv'2021 [paper]
• Multiway Non-rigid Point Cloud Registration via Learned Functional Map Synchronization. arxiv'2021 [paper]
• DeepUME: Learning the Universal Manifold Embedding for Robust Point Cloud Registration. BMVC'2021 [paper]
• Deterministic Point Cloud Registration via Novel Transformation Decomposition. CVPR'2022 [paper]

Fine Registration

The fine registration methods (or local registration) produce highly precise registration results, given the initial pose between two point clouds.

Traditional

• Point2Point ICP: A Method for Registration of 3-D Shapes. TPAMI'1992 [paper]
• Point2Plane Object Modelling by Registration of Multiple Range Images. TPAMI'1992 [paper]
• RPM: New Algorithms for 2D and 3D Point Matching: Pose Estimation and Correspondence. [paper]
• Matching of 3-D Curves using Semi-differential Invariants. ICCV'1995 [paper]
• The Trimmed Iterative Closest Point Algorithm. 2002 [paper]
• Comparing ICP Variants on Real-World Data Sets. 2013 [paper]
• Generalized-ICP. RSS'2009 [paper] [code]
• Go-ICP: Solving 3D Registration Efficiently and Globally Optimally. ICCV'2013 [paper] [code]
• Colored Point Cloud Registration Revisited. ICCV'2017 [paper]
• AA-ICP: Iterative Closest point with Anderson Acceleration. ICRA'2018 [paper]
• Point Clouds Registration with Probabilistic Data Association. IROS'2016 [paper] [code]
• GH-ICP：Iterative Closest Point Algorithm with Global Optimal Matching and Hybrid Metric. 3DV'2018 [paper] [code]
• NDT: The Normal Distributions Transform: A New Approach To Laser Scan Matching. IROS'2003 [paper]
• Best Buddies Registration for Point Clouds. ACCV'2020 [paper]
• Provably Approximated ICP. arxiv'2021 [paper]

Learning-based

• DeepICP: An End-to-End Deep Neural Network for 3D Point Cloud Registration. ICCV'2019 [paper]

Datasets

• Standford 3DScanning
• 3DMatch
• ETH (Challenging data sets for point cloud registration algorithms)
• KITTI Odometry
• ModelNet
• A Benchmark for Point Clouds Registration Algorithms
• WHU-TLS Benchmark
• LiDAR-Registration-Benchmark

Tools

• Open3D: A Modern Libray for 3D Data Processing
• PCL: Point Cloud Library
• Pytorch Geometric
• PyTorch Points 3D
• probreg