search

wenbin-lin / OcclusionFusion

OcclusionFusion: realtime dynamic 3D reconstruction based on single-view RGB-D
https://wenbin-lin.github.io/OcclusionFusion/
256 stars 41 forks source link

Rotation term during post-processing #5

Open shubhMaheshwari opened 2 years ago

shubhMaheshwari commented 2 years ago

Hi @wenbin-lin, Thanks for releasing the motion completion module in OcclusionFusion. I had a doubt regarding the implementation of graph-based ARAP deformation. The paper mentions the use of ARAP term from Embedded Deformation. That it is similar to the post processing applied by 4DComplete.
4D complete uses ARAP on all vertices of the mesh, whereas in OcclusionFusion only graph nodes are used. Hence during optimisation the rotation term will not update. Can you also share the code for post-processing ?

Otherwise can you explain the ARAP in greater detail. Like the loss term, what parameters are updated, number of iterations, etc.
Also during reconstruction is the post processing performed before or after the optimisation?

wenbin-lin commented 2 years ago

The graph-based ARAP deformation is implemented as E_reg in our paper. The energy term for post-processing is a combination of E_motion and E_reg, and we optimize the rigid rotation and translation of each node.

During reconstruction, we do not need to do post-processing since we already include E_reg in the optimization.