This work is based on our ECCV2018 paper. 3DFeat-Net is an approach for learning features for point cloud geometric registration under weak-supervision, where the supervision is given in terms of whether 2 point clouds have very high overlap or low (or no) overlap. For details, please read our paper which can be found on arXiv.
Bibtex:
@inproceedings{yew2018-3dfeatnet,
title={3DFeat-Net: Weakly Supervised Local 3D Features for Point Cloud Registration},
author={Yew, Zi Jian and Lee, Gim Hee},
booktitle={ECCV},
year={2018}
}
Our code is developed and tested on the following environment:
We also use MATLAB scripts for evaluation and processing of data.
The network model is in models/feat3dnet.py
.
Before using the model, you first need to compile the customized tf_ops in the folder tf_ops
(we use the customized grouping and sampling ops from PointNet++).
Check and execute tf_xxx_compile.sh
under each subfolder. Update the python and nvcc file if necessary. The scripts has been updated for TF1.4, so if you're using TF version < 1.4, refer to the original script provided with PointNet++ for compilation.
../data/oxford
, which should contain two subfolders: clusters
and train
.Training is divided into 2 stages, where the first stage only trains the descriptor subnetwork without rotation and attention. For convenience, we provide a training script which runs both parts. Simply execute./train.sh
(you can configure the top few lines to select the GPU, etc).
Training takes around 1-1.5 days to saturate. During training, progress can be monitored by running tensorboard --logdir=./ckpt
from the root folder, and the false alarm rate will be shown in the fp_rate graph.
inference_example.sh
which will load the pretrained model in the folder ckpt
and generate the keypoints and descriptors for the example data in example_data
. A sample checkpoint can be downloaded from here. The output will be stored in example_data/results
.scripts/computeAndVisualizeMatches.m
which will match the features, estimate the relative transformation (with RANSAC) between the point clouds and display the results.It should be straightforward to run on your own data, just make sure the data is in the expected format (see scripts_data_processing/Readme.md
). Note however the following:
--randomize_points
is set which will randomize the input point ordering). This means that the performance may differ slightly with each run.Refer to scripts_data_processing/Readme.md.