search

taconite / PTF

[CVPR'21] Locally Aware Piecewise Transformation Fields for 3D Human Mesh Registration
https://taconite.github.io/PTF/website/PTF.html
MIT License
84 stars 8 forks source link

Unusual PTF Mesh Generation and Registration #12

Open Suraj-Bhor opened 7 months ago

Suraj-Bhor commented 7 months ago

Hello, Firstly, I'd like to express my appreciation for the innovative work done on PTF. It's an impressive tool that I'm excited to explore further. I've been working on SMPL/SMPL+D Registration of meshes using PTF. As mentioned in the readme I have performed all the steps for building the dataset using the steps outlined in the documentation, from 0 to 4, and subsequently generating and registering meshes with the PTF framework. However, I'm encountering an issue where the generated meshes appear unrealistic, which seems to be similar to a previously reported issue. You can view the generated meshes and registered meshes here. (similar output as mentioned earlier )

I suspect that the issue might be related to the scaling of the meshes, even though the current implementation appears to account for scaling factors. To provide a clearer picture, I've attached an image showing two side-by-side meshes.

image

Could you please advise on potential steps I could take to resolve this? I'm wondering if there might be additional considerations or adjustments needed in the scaling.

Also attaching the logs that were generated for registering the mesh.

2024-01-25 12:17:43,450 Inner distance for input shortlong_ATUsquat.000001: 0.3551903963088989 cm 2024-01-25 12:17:43,451 Outer distance for input shortlong_ATUsquat.000001: 0.6012940406799316 cm 2024-01-25 12:17:43,451 Inner distance for input shortlong_ATUsquat.000006: 0.3552056550979614 cm 2024-01-25 12:17:43,451 Outer distance for input shortlong_ATUsquat.000006: 0.6021128296852112 cm 2024-01-25 12:17:43,451 Inner distance for input shortlong_ATUsquat.000011: 0.352639764547348 cm 2024-01-25 12:17:43,451 Outer distance for input shortlong_ATUsquat.000011: 0.5998966693878174 cm 2024-01-25 12:17:43,451 Inner distance for input shortlong_ATUsquat.000016: 0.3565904200077057 cm Output truncated for brevity

Best, Suraj B

Suraj-Bhor commented 6 months ago

Found the root cause of the issue. I was not building the Kaolin package correctly with the correct Cuda environment and hence was facing issues while calculating the triangle distance. After building kaolin correctly, the SMPL registrations looked way more better and realistic.