Open UnrealXinda opened 6 months ago
Same question
I run the examples/demo.py to display SMPL model, but why the Joints shape
is (45, 3) but (24, 3)?
I am curious about the output
I also have this doubt. It would be great if someone could clarify this. Further, is there any joint regressor available which can give me these 45 joints from the SMPL vertices? (I am evaluating an existing method which directly regresses vertex positions. However, to evaluate it with the same script as other methods, I would need to get these 45 joints using a regressor).
Hi, I believe this is related to the information discussed below.
As far as I examined, it appears that the number of joints can be modified using the joint_mapper, which can be configured within each SMPL class. https://github.com/vchoutas/smplx/blob/1265df7ba545e8b00f72e7c557c766e15c71632f/smplx/body_models.py#L489
For example, in SMPLify-X, they've implemented joint_mapper and VertexJointSelector to convert SMPL-X joints and vertices to the OpenPose25 detected keypoints. https://github.com/vchoutas/smplify-x/blob/3e11ff1daed20c88cd00239abf5b9fc7ba856bb6/smplifyx/utils.py#L98
The correspondences between SMPL-X vertex indices and OpenPose keypoint names are defined here. https://github.com/vchoutas/smplx/blob/1265df7ba545e8b00f72e7c557c766e15c71632f/smplx/vertex_ids.py
Basically SMPL-X outputs (body joints + hand joints + face joints, 3), on the other hand SMPLX class with this joint_mapper outputs (25, 3) joints for OpenPose definition. Therefore, the joint_mapper allows for the adjustment of the number of outputted joints to any desired quantity, whether more or less.
I hope this information would be helpful.
Update I figured out why it happens. As I thought, the thing comes from VertexJointSelector.
45 joints = (SMPL joints + face vertices + feet vertices + both hand vertices) = (24 + 5 + 6 + 5*2) https://github.com/vchoutas/smplx/blob/1265df7ba545e8b00f72e7c557c766e15c71632f/smplx/vertex_joint_selector.py#L29
The default argments of VertexJointSelector are set to both use_hands= True, use_hands = True
. On top of that, wheter both are set True
or False
, at least face_keyp_idxs
is added, which means outputed joints should be (24 + 5, 3) joints.
I think just extracting first (24, 3) tensors would be OK in practice.
class VertexJointSelector(nn.Module):
def __init__(self, vertex_ids=None,
use_hands=True,
use_feet_keypoints=True, **kwargs):
super(VertexJointSelector, self).__init__()
extra_joints_idxs = []
face_keyp_idxs = np.array([
vertex_ids['nose'],
vertex_ids['reye'],
vertex_ids['leye'],
vertex_ids['rear'],
vertex_ids['lear']], dtype=np.int64)
extra_joints_idxs = np.concatenate([extra_joints_idxs,
face_keyp_idxs])
if use_feet_keypoints:
feet_keyp_idxs = np.array([vertex_ids['LBigToe'],
vertex_ids['LSmallToe'],
vertex_ids['LHeel'],
vertex_ids['RBigToe'],
vertex_ids['RSmallToe'],
vertex_ids['RHeel']], dtype=np.int32)
extra_joints_idxs = np.concatenate(
[extra_joints_idxs, feet_keyp_idxs])
if use_hands:
self.tip_names = ['thumb', 'index', 'middle', 'ring', 'pinky']
tips_idxs = []
for hand_id in ['l', 'r']:
for tip_name in self.tip_names:
tips_idxs.append(vertex_ids[hand_id + tip_name])
extra_joints_idxs = np.concatenate(
[extra_joints_idxs, tips_idxs])
self.register_buffer('extra_joints_idxs',
to_tensor(extra_joints_idxs, dtype=torch.long))
Hi, could someone help me with interpreting the results of SMPLOutput? I can get a rough estimate of what each field means. But I can't seme to find documentation on what coordinate space those values are under. Specifically, are they in world space? Component space (wrt to the character's root or whatever fixed point it is)? Local space (with relative to its parent bone/joint's coordinate system)? The fields that I'm trying to understand are:
Thank you!