How to get cocoplus_regressor?

[Yuliang-Zou]

Hi, I wonder how did you get the parameters for the cocoplus_regressor? Some joints are shared by both SMPL and cocoplus, I check that row of 6890-d vectors from J_regressor and cocoplus_regressor, and found that they are different (cocoplus_regressor seems to have fewer valid vertices for each joint). So I am quite curious how you computed the parameters for cocoplus_regressor (for the last five I know that you just directly pick the corresponding vertex).

[akanazawa]

Hi,

This was done in SMPLify, see page 10 last paragraph. Basically on training Moshed Human3.6M, we have both the SMPL mesh and the 3D joints of Human3.6M. So we learned a linear regressor that goes from vertices to those 3D joints.

Best,

Angjoo

[JiangLiguo]

Hi, Thanks you for sharing wonderful works, and I had seen page 10 last paragraph in SMPLify, but there are few description for Joint regressor in SMPLify. Can you provide more details about it? Best.

[akanazawa]

Hi JiangLiguo,

So, if you have Moshed results on a mocap dataset (i.e. AMASS), and if the mocap dataset comes with it's own definition of 3D joints (as they often do but every dataset may have a different definition), then you basically have many examples of 3D mesh vertices (coming from the moshed params) & the 3D joints. Then you just learn a linear regressor that goes from N x 3 to |J| x 3, where |J| is the number of joints, and N >> |J| in general so it's not a hard problem to solve.

For coco-plus, we also added keypoints on the faces like the eyes. Those we did by picking a vertex that correspond to the keypoint (like left eye is vertex id x), this is appened to the learned regression matrix as a one-hot connection.

Hope this helps,

A

[JiangLiguo]

Hi, Thanks you for your quick reply. The linear regressor takes all 3D mesh vertices as input or whether some 3D mesh vertices which are picked manually. Best.

[JiangLiguo]

Hi, I am preparing a paper for Siggraph in next month. But we adopt an new parametric model without using SMPL, so I need learn an linear regressor that goes from mesh vertices to joints.

[akanazawa]

Aside from the face points, where vertex assignment is clear, It takes all 3D vertices, and learns which linear combination of vertices to use to get the semantic 3D keypoints.

On Thu, Dec 19, 2019 at 11:18 AM JiangLiguo notifications@github.com wrote:

Hi, I am preparing a paper for Siggraph in next month. But we adopt an new parametric model without using SMPL, so I need learn an linear regressor that goes from mesh vertices to joints.

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/akanazawa/hmr/issues/70?email_source=notifications&email_token=AAHBXWH4VDF7LGF4HPKXUMDQZONGZA5CNFSM4GYQ7F52YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEHKD6GY#issuecomment-567557915, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAHBXWBTFO57HUS3NF2FSODQZONGZANCNFSM4GYQ7F5Q .

[jiangwei221]

Hi, A dumb question: where can I download the cocoplus_regressor weights? Thanks in advance.

[akanazawa]

I forget since it's been a while but I think it's in here https://github.com/akanazawa/hmr/blob/master/doc/train.md see: that last .pkl is the regressor

[jiangwei221]

Thanks! I found it here: wget https://people.eecs.berkeley.edu/~kanazawa/cachedir/hmr/models.tar.gz && tar -xf models.tar.gz