search

chungyiweng / humannerf

HumanNeRF turns a monocular video of moving people into a 360 free-viewpoint video.
MIT License
786 stars 86 forks source link

animate model #19

Closed Andyen512 closed 2 years ago

Andyen512 commented 2 years ago

Can the trained HumanNerf model be used to generate some new unseen poses?

chungyiweng commented 2 years ago

Yes, it has the capability of being applied to unseen poses but we did not evaluate it carefully as it's not our focus.

mgholamikn commented 2 years ago

Thank you for sharing the code. Could you please let me know how I can give random 3D poses (smpl param) as input to the model?

Erickrus commented 2 years ago

Render unseen pose

If you want to change T-pose to an unseen pose (let's say Y-pose), you can reference SMPL FAQ P23. Find out the corresponding joint ids for both shoulders. And then in core/data/human_nerf/tpose.py line 140. Add 2 new lines:

dst_poses[16*3+2] =  1.0 # Left shoulder  rotate +1/pi*180 = +57 deg
dst_poses[17*3+2] = -1.0 # right shoulder rotate -1/pi*180 = -57 deg

Notice, some parts of armpit are not clearly rendered, I guess, due to the sampling.

Have fun!

https://user-images.githubusercontent.com/1734553/187606515-08c02199-78c5-4068-b8e8-5565c8c4ae56.mp4