π SGHR
Haiping Wang*,1,
Yuan Liu*,2,
Zhen Dong†,1,
Yulan Guo3,
Yu-Shen Liu4,
Wenping Wang5
Bisheng Yang†,1
1Wuhan University
2The University of Hong Kong
3Sun Yat-sen University
4Tsinghua University
5Texas A&M University
*The first two authors contribute equally.
†Corresponding authors.
In this paper, we present a new method for the multiview registration of point cloud. Previous multiview registration methods rely on exhaustive pairwise registration to construct a densely-connected pose graph and apply Iteratively Reweighted Least Square (IRLS) on the pose graph to compute the scan poses. However, constructing a densely-connected graph is time-consuming and contains lots of outlier edges, which makes the subsequent IRLS struggle to find correct poses. To address the above problems, we first propose to use a neural network to estimate the overlap between scan pairs, which enables us to construct a sparse but reliable pose graph. Then, we design a novel history reweighting function in the IRLS scheme, which has strong robustness to outlier edges on the graph. In comparison with existing multiview registration methods, our method achieves $11$\% higher registration recall on the 3DMatch dataset and $\sim13$\% lower registration errors on the ScanNet dataset while reducing $\sim70$\% required pairwise registrations. Comprehensive ablation studies are conducted to demonstrate the effectiveness of our designs.
Here we offer the YOHO backbone SGHR. Thus YOHO requirements need to be met:
Specifically, The code has been tested with:
First, create the conda environment:
conda create -n sghr python=3.7
conda activate sghr
Second, intall Pytorch. We have checked version 1.7.1 and other versions can be referred to Official Set.
conda install pytorch==1.7.1 torchvision==0.8.2 torchaudio==0.7.2 cudatoolkit=11.0 -c pytorch
Third, install other packages, here we use 0.8.0.0 version Open3d for Ubuntu 16.04:
pip install -r requirements.txt
Optional. If you want to use SGHR on your own dataset or run the demo.py, you should install MinkowskiEngine for FCGF/YOHO:
conda install openblas-devel -c anaconda
git clone https://github.com/NVIDIA/MinkowskiEngine.git
cd MinkowskiEngine
python setup.py install --blas_include_dirs=${CONDA_PREFIX}/include --blas=openblas
The datasets are accessible in BaiduDesk(Code:oouk) and Google Cloud:
Trainset:
Testset:
Datasets above contain the point clouds (.ply), keypoints (.txt, 5000 per point cloud), and rotation-invariant yoho-desc(.npy, extracted on the keypoints) files. Please place the data to ./data
following the example data structure as:
data/
βββ 3dmatch/
βββ kitchen/
βββ PointCloud/
βββ cloud_bin_0.ply
βββ gt.log
βββ gt.info
βββ yoho_desc/
βββ 0.npy
βββ Keypoints/
βββ cloud_bin_0Keypoints.txt
βββ 3dmatch_train/
βββ scannet/
βββ ETH/
You can train SGHR with the 3dmatch_train dataset downloaded above, where we offer the 32-dim rotation-invariant yoho-desc we extracted on 3dmatch_train and you can also extract 32-dim invariant yoho-desc(row-pooling on yoho-desc) yourself and save the features to '''data/3dmatch_train/\<scene>/yoho_desc'''. Then, you can train SGHR with the following commond:
python Train.py
Use SGHR is quite simple, prepare your point cloud files and no other effort needed! Follow here.
Try SGHR on the demo files by:
python demo.py --pcdir data/demo
To evalute SGHR on 3DMatch and 3DLoMatch, you can use the following commands:
# extract global features
python Test.py --dataset 3dmatch
# conduct multiview registration
python Test_cycle.py --dataset 3dmatch --rr
# visualize the registration results
python visual.py --dataset 3dmatch
To evalute SGHR on ScanNet, you can use the following commands:
python Test.py --dataset scannet
python Test_cycle.py --dataset scannet --ecdf
To evalute SGHR on ETH, you can use the following commands:
python Test.py --dataset ETH
python Test_cycle.py --dataset ETH --topk 6 --inlierd 0.2 --tau_2 0.5 --rr
Please consider citing SGHR if this program benefits your project
@inproceedings{
wang2023robust,
title={Robust Multiview Point Cloud Registration with Reliable Pose Graph Initialization and History Reweighting},
author={Haiping Wang and Yuan Liu and Zhen Dong and Yulan Guo and Yu-Shen Liu and Wenping Wang and Bisheng Yang},
booktitle={Conference on Computer Vision and Pattern Recognition},
year={2023}
}
Take a look at our previous works on feature extraction and pairwise registration!