search

akanazawa / hmr

Project page for End-to-end Recovery of Human Shape and Pose
Other
1.55k stars 395 forks source link

The question about running HMR on a video sequence #66

Closed YuQingtao closed 5 years ago

YuQingtao commented 5 years ago

Hi, I ran HMR on a video sequence and also found the motion was not smooth. I read the paper "SFV: Reinforcement Learning of Physical Skills from Videos" to find the solution. But I still have some questions. I want to ask what method you use to optimize. And I don't know how to use the q fuction of q(z) because it is a part of the network. So do you still use the neural network to optimize the features z? Thanks, Yuqingtao

daydreamer2023 commented 5 years ago

the latest paper&project. Learning 3D Human Dynamics from Video https://akanazawa.github.io/human_dynamics/# https://arxiv.org/abs/1812.01601

akanazawa commented 5 years ago

Hi,

The idea is very similar to SMPLify, or you can think of it was bundle adjustment / iterative optimization over the input sequence. Basically you take HMR output as initialization, and then solve for latent parameter (in SfV it was the final 2048D image features that we solve for) that minimizes the reprojection loss with detected 2D joints + smoothness losses. You could've also optimized in the beta and theta space. I have been meaning to make this code public. I have a draft version that I could share with you and you could try it out if you send me an email.

Thanks,

Angjoo