Closed ZhaoLizz closed 7 months ago
Thanks for your interest in our work. I'm sorry that the J meaning in the code may be a little misleading since I modified a little bit the SMPLX code for J, modified here. When you look into this function, J
is actually meaning the shape-dependent joint position in the canonical space and J_transformed
is the transformed joint position, which means J is not in da-pose, but in zero-pose.
So during initialization, self.J_canonical
is still in the zero-pose, but only affected by the initial shape beta
. Then, during forward, _, A = batch_rigid_transform( axis_angle_to_matrix(theta), self.J_canonical.expand(nB, -1, -1), self._template_layer.parents, )
is a standard forward where A
transforms all the points in zero pose to the current target pose. However, remember during initialization, all the Gaussians are initialized on the da-pose mesh (which means that the canonical Gaussian centers are on da-pose, not zero T-pose).
Because the A
you compute with the batch_rigid_transform
is to transform the point in zero pose to the current pose, but the canonical Gaussian centers are on da-pose. We have to first transform the canonical center back to zero pose and then transform to current pose, that's why the A0_inv
is saved during initialization and right multiplicated -- to first transform the da-pose center back to zero pose and then use batch_rigid_transform
to go to current pose.
I appreciate your detailed reply! My question has been perfectly answered!
First and foremost, I'd like to express my appreciation for the outstanding work. I have a query regarding a specific part of the implementation.
In the forward function of SMPLTemplate, there's a line where the transformation matrix A is calculated as follows:
A = torch.einsum("bnij, njk->bnik", A, self.A0_inv)
I'm seeking clarification on the conceptual meaning behind the operationA dot A0_inv
. As I understand it,A0
represents the relative transformation from the SMPL default pose joints (J
) to the DaPose joints (J_dapose
). Consequently,inv(A0)
should denote the transformation fromJ_dapose
back toJ
.If my understanding is correct, then
A
in this context represents the transformation fromJ_dapose
to the joints in the theta pose (J_theta
). Could you please confirm if my interpretation is accurate or provide further explanation ofA dot A0_inv
if necessary?Thank you in advance for your assistance!
Relevant code are attached here for convenience.