2D, Non-Rigid | 3D, Rigid |
---|---|
This website hosts implementations of the robust point set registration framework described in the paper "Robust Point Set Registration Using Gaussian Mixture Models", Bing Jian and Baba C. Vemuri, IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011, 33(8), pp. 1633-1645. An earlier conference version of this work, "A Robust Algorithm for Point Set Registration Using Mixture of Gaussians, Bing Jian and Baba C. Vemuri.", appeared in the proceedings of ICCV'05. Here is the bibtex entry for citing this work.
A python implementation (unoptimized and mainly for proof of concept) can be found in http://github.com/bing-jian/gmmreg-python.
Recently, a group of researchers from Estonia and Spain reported an interesting work of garment retexturing in a paper titled "From 2D to 3D geodesic-based garment matching". The GMM based point set registration algorithm was chosen by them for contour matching, a critical component in their method for garment retexturing. The following comments are taken from Section 3.2 in their paper:
"Out of available algorithms, we have chosen to use non-rigid point set registration using Gaussian mixture models (GMM) [21] because of its accurate fitting under different conditions and fast execution time. Additionally, Gaussian mixtures provide robust results even if the shapes have different features, such as different neck lines, hand positions and folds."
In SIGGRAPH Asia 2016, Yan et al. proposed a method for global registration of building scans. Our GMM based point set registration algorithm was used for pairwise portal matching in their pipeline as indicated in Section 4.3 of their paper.
The GMM based point set registration algorithm was used in "3-D Vessel Tree Surface Reconstruction", a patent (United States Patent 10140733) filed by researchers at Siemens Corporate Research in Princeton, as shown in Eq(2) and Eq(16).
The basic idea of the proposed point set registration framework is to 1) represent the two point sets by continuous distributions, in particular, Gaussian mixture models; 2) and then minimize the distance between the two distributions by moving one towards another. Interestingly, several previous well-known point set registration algorithms can all be re-formulated using this unified framework, including:
[Iterative Closest Point (ICP)] Besl, Paul J.; N.D. McKay (1992). "A Method for Registration of 3-D Shapes". IEEE Transactions on Pattern Analysis and Machine Intelligence. 14 (2): 239–256.
[TPS-RPM] Haili Chui and Anand Rangarajan, A new point matching algorithm for non-rigid registration, Computer Vision and Image Understanding, 2003, 89(2-3), pp. 114-141.
[KC-Reg] Yanghai Tsin and Takeo Kanade, A Correlation-Based Approach to Robust Point Set Registration, ECCV (3) 2004: 558-569.
[GMMREG-L2] Bing Jian and Baba C. Vemuri, A Robust Algorithm for Point Set Registration Using Mixture of Gaussians, ICCV 2005, pp. 1246-1251.
[Coherent Point Drift (CPD)] Andriy Myronenko, Xubo B. Song, Miguel A. Carreira-Perpinan, Non-rigid Point Set Registration: Coherent Point Drift, NIPS 2006, pp. 1009-1016.
Please see Fig 3. in Jian&Vemuri PAMI'11 on how those methods can be reformulated under this unified framework by choosing different statistical divergence functions.
For a growing list of recent work on point set registration/matching, please refer to this actively maintained github repo.
To build the execuatables from the source code, please use CMake. Please note that the current C++ implementation depends on vxl/vnl for doing matrix computation and numerical optimization.
The executable (named "gmmreg_demo") takes a configuration file (in INI format) and a tag string from command line. For more on usage, please check this file. For examples of config INI file, please check this folder.
Test non-rigid registration.
git clone https://github.com/bing-jian/gmmreg.git --recursive
cd C++
mkdir build && cd build
cmake -DCMAKE_BUILD_TYPE=Release ..
make
cd ../../data
../C++/build/gmmreg_demo ./face.ini tps_l2
../C++/build/gmmreg_demo ./fish_full.ini tps_l2
../C++/build/gmmreg_demo ./fish_partial.ini tps_l2
Test 3D rigid registration on a 3D scan dataset.
Test 3D rigid registration on a RGB-D dataset.
Before | Registration | After |
---|---|---|
Transformation estimated by gmmreg:
[[ 0.979206 -0.0376796 0.199341 -0.174986 ]
[ 0.0348325 0.999235 0.0177716 0.106985 ]
[-0.199858 -0.0104585 0.979769 -0.191445 ]
[ 0. 0. 0. 1. ]]
Transformation from ground truth:
[[ 0.98046985 -0.0365099 0.19324962 -0.16860252]
[ 0.03337988 0.99925376 0.01942979 0.10915275]
[-0.19381476 -0.01259967 0.9809579 -0.19450066]
[ 0. 0. 0. 1. ]]
('pose difference (in degrees) before alignment:', 11.379813087519903)
('pose difference (in degrees) after alignment:', 0.37620688052421786)
Test environment and setting
Results on 295 pairs (every 5 frames from the first 300 frames, e.g. registering i-th and (i+5)-th for i from 1 to 295).
The avg pair-wise pose difference among these 295 pairs is ~4.84 degree.
The median angle error against the ground truth is ~0.6 degree and the median fps is ~8 with Intel i7-6850K CPU.
Metric | Avg | Min | Max | Median |
---|---|---|---|---|
Rotation angle error (in degrees) | 0.96 | 0.07 | 9.08 | 0.60 |
Run time per pair (in milliseconds) | 120.14 | 47.09 | 413.48 | 116.62 |
Metric | Avg | Min | Max | Median |
---|---|---|---|---|
Rotation angle error (in degrees) | 0.65 | 0.01 | 19.49 | 0.41 |
Run time per pair (in milliseconds) | 76.18 | 18.27 | 508.25 | 60.06 |