search

songrise / AvatarCraft

[ICCV23] AvatarCraft: Transforming Text into Neural Human Avatars with Parameterized Shape and Pose Control
Other
186 stars 7 forks source link

Confusion about the Avatar Creation Results #15

Closed duyanxi closed 8 months ago

duyanxi commented 8 months ago

Your work AvatarCraft impressed me a lot and it inspired me a lot.

But when I use the following command to create an avatar with text prompt, the results are far from expectation. python stylize.py --weights_path "ckpts/bare_smpl.pth.tar" --tgt_text "Hulk, photorealistic style" --exp_name "hulk" --batch_size 4096 15376753f03bffa79c658959ada4d84 Here shows the canonical render result: exp_body_can

Are these results normal? Also, we found there seemed to be a mode collapse after 4800iter. Can you help with this problem?

Ps. We test this code on Win10+cuda 11.1 and RTX3090-24G, and we download the stable-diffusion from huggingface instead of importing it online.

songrise commented 8 months ago

Hi, those results are abnormal. We did not encounter results like this. To clarify, it is possible to get mode-collapse in AvatarCraft with some particular prompts, but the figures you show appears to be another issue. It looks like the nerf is not properly initialized, or the reg term set too low. Could you please provide the render results from early steps (0 - 2000)?

duyanxi commented 8 months ago

Thank you for your quick response. This is our render result from step 2000. We found that the result from step 4800(stage 1)is better, but the final result maybe have some problem. exp_body_can

songrise commented 8 months ago

Thanks for your reply. The result is strange, and my guess is that the network is not initialized with mesh reconstruction but random initilization. Could you please check if the render result in step 0 is a human body or not?

duyanxi commented 8 months ago

You are right. We check the result from result0 and found it is not a human body. zombie_0001_body In fact, it looks like this. How can we check the initialization of the network in the code?

duyanxi commented 8 months ago

Thank you for your help! We've fixed our bug. It's only a simple path error caused by our carelessness. Thanks for your quick response again and this is an excellent work!