Meaning of (A dot A0_inv) of SMPLTemplate

[ZhaoLizz]

First and foremost, I'd like to express my appreciation for the outstanding work. I have a query regarding a specific part of the implementation.

In the forward function of SMPLTemplate, there's a line where the transformation matrix A is calculated as follows: A = torch.einsum("bnij, njk->bnik", A, self.A0_inv) I'm seeking clarification on the conceptual meaning behind the operation A dot A0_inv. As I understand it, A0 represents the relative transformation from the SMPL default pose joints (J) to the DaPose joints (J_dapose). Consequently, inv(A0) should denote the transformation from J_dapose back to J.

If my understanding is correct, then A in this context represents the transformation from J_dapose to the joints in the theta pose (J_theta). Could you please confirm if my interpretation is accurate or provide further explanation of A dot A0_inv if necessary?

Thank you in advance for your assistance!

Relevant code are attached here for convenience.

init_smpl_output = self._template_layer(
    betas=init_beta[None],
    body_pose=can_pose[None, 1:],
    global_orient=can_pose[None, 0],
    return_full_pose=True,
)
J_canonical, A0 = init_smpl_output.J, init_smpl_output.A
A0_inv = torch.inverse(A0)

def forward(self, theta=None, xyz_canonical=None):
            # skinning
            _, A = batch_rigid_transform(
                axis_angle_to_matrix(theta),
                self.J_canonical.expand(nB, -1, -1),
                self._template_layer.parents,
            )
            A = torch.einsum("bnij, njk->bnik", A, self.A0_inv) 

[JiahuiLei]

Thanks for your interest in our work. I'm sorry that the J meaning in the code may be a little misleading since I modified a little bit the SMPLX code for J, modified here. When you look into this function, J is actually meaning the shape-dependent joint position in the canonical space and J_transformed is the transformed joint position, which means J is not in da-pose, but in zero-pose.

So during initialization, self.J_canonical is still in the zero-pose, but only affected by the initial shape beta. Then, during forward, _, A = batch_rigid_transform( axis_angle_to_matrix(theta), self.J_canonical.expand(nB, -1, -1), self._template_layer.parents, ) is a standard forward where A transforms all the points in zero pose to the current target pose. However, remember during initialization, all the Gaussians are initialized on the da-pose mesh (which means that the canonical Gaussian centers are on da-pose, not zero T-pose). Because the A you compute with the batch_rigid_transform is to transform the point in zero pose to the current pose, but the canonical Gaussian centers are on da-pose. We have to first transform the canonical center back to zero pose and then transform to current pose, that's why the A0_inv is saved during initialization and right multiplicated -- to first transform the da-pose center back to zero pose and then use batch_rigid_transform to go to current pose.

[ZhaoLizz]

I appreciate your detailed reply! My question has been perfectly answered!