bharat-b7 / RVH_Mesh_Registration

Code to fit SMPL model to scans
179 stars 21 forks source link

What is the best mesh registration algorithm? #14

Closed BioR0bot closed 1 year ago

xiexh20 commented 1 year ago

That is an interesting question and the answer depends on your needs and input quality.

Currently there are 4 registration methods listed here. They are roughly ranked according to the noise and difficulty of registrations as well as required additional inputs.

  1. The first registration is suitable for clean scans with simple poses. The SMPL pose is initialized from zero pose hence if the scan pose does not deviate a lot from zero pose (e.g. A-pose), then this code would work reasonably.
  2. The second registration simply adds one deformation layer to the SMPL(H) model, which models the clothing. Similar to the first one, it only handles clean and simple poses.
  3. The third registration deals with point cloud inputs, which means less clear renderings and more noisy openpose detections. Hence we recommend using additional pose prediction from FrankMocap to initialize the registration. Otherwise it is difficult to fit from zero pose.
  4. The forth method is based on IP-Net, which is a learning based approach designed to replace the tedious rendering and openpose detections in previous methods. However, its performance largely depends on the training data. So if your input data deviates a lot from the training data, then the prediction won't be very accurate. Currently released IP-Net model is trained on clean scans, hence its prediction might work better than method 3 if your point clouds are very noisy.