search

aipixel / GaussianAvatar

[CVPR 2024] The official repo for "GaussianAvatar: Towards Realistic Human Avatar Modeling from a Single Video via Animatable 3D Gaussians"
https://huliangxiao.github.io/GaussianAvatar
MIT License
343 stars 22 forks source link

Error poses after training stage1 #13

Open LinzhouLi opened 4 months ago

LinzhouLi commented 4 months ago

Hi, thanks for your great work!

I trained GaussianAvatar with dynvideo_female and dynvideo_male sequences. I found there are some artifacts of smpl poses after stage1 training. The optimized poses are even worse than the origin poses. I would like to ask what cause this issue?

Below are some example of the error poses. (from left to right: GT image, origin pose, optimized pose after stage1) 00157 00542 00039 00388

Best regards!

huliangxiao commented 4 months ago

Hey there! I recommend you to take a look at the following:

1)Double-check whether it's SMPL-X or SMPL optimization. 2)Verify the corresponding masks as well.

LinzhouLi commented 4 months ago

Thanks for your kind advice!

1) I'm sure about I use SMPL optimization and the initial SMPL parameters are provided by your dataset. And I even delete smplx in assets folder to make sure I only use SMPL information for optimization.

2) I retrain the model with masks provided by your dataset. And rename them to match the files in images folder. I'll show they are matched in the figures below.

After that, the error poses after optimation still exist.

Below are 3 error poses. (from left to right: GT image, GT mask, origin pose, optimized pose after stage1)

00159 00341 00542

huliangxiao commented 4 months ago

Hi, here are some results after retraining: 022211190435_0199image

LinzhouLi commented 4 months ago

Thanks! I know that most of poses are correct after optimization, but some error poses appear. Could you please release the trained model after stage 1?