yanchxx / MoPA

An Blender addon uses ROMP to extract human's 3D poses from image, video or webcam and drive your own 3D character.
MIT License
242 stars 49 forks source link

Detection variable: outputs = {'poses': poses, 'trans': trans[0]} #27

Closed zhangby2085 closed 1 year ago

zhangby2085 commented 1 year ago

Hey, Yanchxx,

May I ask, what is the output variable, I think "trans" is the translation between camera and object.

What are the poses, is it the location x, y, z, or rotation x, y, z, degrees?

Thank you 👍

yanchxx commented 1 year ago

"trans" is the hip's position in the world coordinate "poses" is joint's rotation in axis-angle representation

zhangby2085 commented 1 year ago

Many thanks for the reply, it helps a lot, I try to get an overall point of view on how it works.

By the way, in the process of running romp, what is the key of params_pred, smpl_thetas, smpl_betas? unable to locate the documentation of them.

yanchxx commented 1 year ago

smpl_thetas, smpl_betas is the term used in paper SMPL

ROMP is a frontier paper code implementation, you can consider to see the relevant knowledge of the paper, obviously there will not be a perfect manual for reference.

foocker commented 1 year ago

will add more frontier paper which based on SMPL on 3d human construction, such as VIBE ,HybrIK and so on. may there are some subtle difference?

zhangby2085 commented 1 year ago

Excellent, great thanks for the effort, the paper and repos are nice to know, wanna extract the bones mapping (location and rotations), is there a way to get from romp? image

yanchxx commented 1 year ago

Maybe this file is helpful. Look at this function.

yanchxx commented 1 year ago

will add more frontier paper which based on SMPL on 3d human construction, such as VIBE ,HybrIK and so on. may there are some subtle difference?

I don't know exactly what you mean? Do you want me to add a feature about VIBE?

zhangby2085 commented 1 year ago

Hello again, and the recommendations are beneficial and helpful, and how about in Unity, I noticed that the difference between Blender and Unity is the axis angle rotations work straight away in Blender, and in Unity, it is differently allocated MicrosoftTeams-image

foocker commented 1 year ago

will add more frontier paper which based on SMPL on 3d human construction, such as VIBE ,HybrIK and so on. may there are some subtle difference?

I don't know exactly what you mean? Do you want me to add a feature about VIBE?

api such as romp output, like vibe...you can choose use which model's output. more api(interface)may can test more new paper's result ,can get better 3d pose infomation to driver. your WOC will open? especial on MeshCNN to autorig.

yanchxx commented 1 year ago

will add more frontier paper which based on SMPL on 3d human construction, such as VIBE ,HybrIK and so on. may there are some subtle difference?

I don't know exactly what you mean? Do you want me to add a feature about VIBE?

api such as romp output, like vibe...you can choose use which model's output. more api(interface)may can test more new paper's result ,can get better 3d pose infomation to driver. your WOC will open? especial on MeshCNN to autorig.

WOC will not open because the company won't allow it. The auto-rig part needs improvement.

Thank you for your advice. I'll be adding some useful features to this repository over the next few years to help visualize more new paper's result.