search

IDEA-Research / DreamWaltz

[NeurIPS 2023] Official implementation of the paper "DreamWaltz: Make a Scene with Complex 3D Animatable Avatars".
https://idea-research.github.io/DreamWaltz/
Other
180 stars 9 forks source link

SMPL Initialization #7

Closed huanngzh closed 9 months ago

huanngzh commented 9 months ago

Why is the depth map used to initialize a NeRF of canonical SMPL? Use color map rendered from mesh will be efficient.

Yukun-Huang commented 9 months ago

Hi, using color map to initialize NeRF brings unnecessary color priors, e.g. white foreground. We tend to initialize NeRF's geometry without introducing appearance preferences.

Yukun-Huang commented 9 months ago

Thank you so much for the threestudio implementation of DreamWaltz! We really appreciate it.

huanngzh commented 9 months ago

Well, got it. Btw, I'm honored to implement such a good avatar generation work as an extension of threestudio.

huanngzh commented 9 months ago

During the implementation process, it was discovered that when your code exported the mesh, depth, and pose maps of SMPL, the rendered mesh were inconsistent with the other two forms in the same camera input, as shown in the figure below. err_pose Could you please let me submit a pull request to resolve this issue?

Yukun-Huang commented 9 months ago

Sure! Thanks for the bug report and please feel free to submit a pull request.