🤗 RoReg
Haiping Wang*,1,
Yuan Liu*,2,
Qingyong Hu3,
Bing Wang4,
Jianguo Chen5,
Zhen Dong†,1,
Yulan Guo6,
Wenping Wang7
Bisheng Yang†,1
1Wuhan University
2The University of Hong Kong
3University of Oxford
4The Hong Kong Polytechnic University
5DiDi Chuxing
6Sun Yat-sen University
7Texas A&M University
*The first two authors contribute equally.
†Corresponding authors.
We present RoReg, a novel point cloud registration framework that fully exploits oriented descriptors and estimated local rotations in the whole registration pipeline. Previous methods mainly focus on extracting rotation-invariant descriptors for registration but unanimously neglect the orientations of descriptors. In this paper, we show that the oriented descriptors and the estimated local rotations are very useful in the whole registration pipeline, including feature description, feature detection, feature matching, and transformation estimation. Consequently, we design a novel oriented descriptor RoReg-Desc and apply RoReg-Desc to estimate the local rotations. Such estimated local rotations enable us to develop a rotation-guided detector, a rotation coherence matcher, and a one-shot-estimation RANSAC, all of which greatly improve the registration performance. Extensive experiments demonstrate that RoReg achieves state-of-the-art performance on the widely-used 3DMatch and 3DLoMatch datasets, and also generalizes well to the outdoor ETH dataset. In particular, we also provide in-depth analysis on each component of RoReg, validating the improvements brought by oriented descriptors and the estimated local rotations.
2023-02-05 RoReg has been accepted by IEEE TPAMI! 🎉🎉
Here we offer the FCGF backbone RoReg. Thus FCGF requirements need to be met:
Specifically, The code has been tested with:
conda create -n roreg python=3.7
conda activate roregconda install pytorch==1.7.1 torchvision==0.8.2 torchaudio==0.7.2 cudatoolkit=11.0 -c pytorchcd utils/MinkowskiEngine
conda install openblas-devel -c anaconda
export CUDA_HOME=/usr/local/cuda-11.1 #We have checked cuda-11.1.
python setup.py install --blas_include_dirs=${CONDA_PREFIX}/include --blas=openblas
cd ../..pip install -r requirements.txtThe datasets and pretrained weights have been uploaded to Google Cloud:
Also, all data above can be downloaded in BaiduDisk(Code:b4zd), where the checkpoints of RoReg and 3dm_train_rot are saved in YOHO/RoReg.
Datasets above contain the point clouds (.ply) and keypoints (.txt, 5000 per point cloud) files. Please place the data to ./data/origin_data following the example data structure as:
data/
├── origin_data/
├── 3dmatch/
└── kitchen/
├── PointCloud/
├── cloud_bin_0.ply
├── gt.log
└── gt.info
└── Keypoints/
└── cloud_bin_0Keypoints.txt
├── 3dmatch_train/
└── ETH/To train RoReg-Desc and local rotation estimation (one-shot transformation estimation) with the FCGF backbone we offer, you can first prepare the trainset:
python trainset.py --component GFand conduct training of the two components by:
python Train.py --component GF # for RoReg-Desc, requiring ~250G storage space.
python Train.py --component ET # for local rotationsAfter the training of RoReg-Desc and local rotation estimation, you can follow the commonds to train rotation-guided detector yourself:
python trainset.py --component RD
python Train.py --component RDTo train rotation coherence matcher yourself, you can follow the commonds of
python trainset.py --component RM # require ~300G storage space.
python Train.py --component RMAll models will be saved in ./checkpoints/FCGF.
With the pretrained models, you can try RoReg with:
python demo.pyWith the TestData downloaded above, the test on 3DMatch and 3DLoMatch can be done by
python testset.py --dataset 3dmatch --voxel_size 0.025python Test.py --RD --RM --ET yohoo --keynum 1000 --testset 3dmatch--RD denotes using the proposed rotation-guided detector, and we will use randomly-sampling without it.
--RM denotes using the proposed rotation coherence matcher, and we will use NN+mutual without it.
--ET denotes the choice of transformation estimation. yohoo means using the proposed one-shot transformation estimation, and we also offer another faster RANSAC variant yohoc.
--keynum denotes to sample how many keypoints in each scan for matching.
--dataset denotes the evaluation dataset: 3dmatch for the 3DMatch dataset and 3dLomatch for the 3DLoMatch dataset.
More options as well as their descriptions can be found in Test.py.
All the results will be placed to ./data/YOHO_FCGF.
The generalization results on the outdoor ETH dataset can be got as follows:
python testset.py --dataset ETH --voxel_size 0.15python Test.py --RD --RM --ET yohoo --keynum 1000 --testset ETH --tau_2 0.2 --tau_3 0.5 --ransac_ird 0.5All the results will be placed to ./data/YOHO_FCGF.
If you find RoReg/YOHO useful in your research, please consider citing:
@inproceedings{wang2022you,
title={You only hypothesize once: Point cloud registration with rotation-equivariant descriptors},
author={Wang, Haiping and Liu, Yuan and Dong, Zhen and Wang, Wenping},
booktitle={Proceedings of the 30th ACM International Conference on Multimedia},
pages={1630--1641},
year={2022}
@ARTICLE{wang2023roreg,
author={Wang, Haiping and Liu, Yuan and Hu, Qingyong and Wang, Bing and Chen, Jianguo and Dong, Zhen and Guo, Yulan and Wang, Wenping and Yang, Bisheng},
journal={IEEE Transactions on Pattern Analysis and Machine Intelligence},
title={RoReg: Pairwise Point Cloud Registration with Oriented Descriptors and Local Rotations},
year={2023},
volume={},
number={},
pages={1-18},
doi={10.1109/TPAMI.2023.3244951}}
}Welcome to take a look at the homepage of our research group WHU-USI3DV ! We focus on 3D Computer Vision, particularly including 3D reconstruction, scene understanding, point cloud processing as well as their applications in intelligent transportation system, digital twin cities, urban sustainable development, and robotics.
YOHO is the early work of RoReg and we sincerely thank the contributing projects: