question about the local_rotation

[Greenpengrk]

Hello, Thanks for your impressive work! I have some questions about the code.

In lines 1148-1154 of time_utils.py, node_attrs['local_rotation'] is used to obtain translation.

if self.local_frame:
    local_rot = node_attrs['local_rotation'] + rot_bias
    local_rot_matrix = quaternion_to_matrix(local_rot)
    nn_nodes = self.nodes[nn_idx,...,:3].detach()
    Ax = torch.einsum('nkab,nkb->nka', local_rot_matrix[nn_idx], x[:, None] - nn_nodes) + nn_nodes + node_trans[nn_idx]
    Ax_avg = (Ax * nn_weight[..., None]).sum(dim=1)
    translate = Ax_avg - x

I noticed that both local_rotation and d_rotation are obtained from the query_network. Could you explain what each of these represents? According to my understanding, d_rotation represents the rotation change of control points from the canonical space to the observation space. Why is local_rotation used here instead of directly using the d_trans?

Any advice or pointers on this would be greatly appreciated. Thank you!

[yihua7]

Hi,

local_rotation affects only the translation of the Gaussian positions. It represents the rotation component of the 6DoF transformation. The interpolation follows a Linear Blend Skinning (LBS) method.

d_rotation is the residual value of the Gaussian rotation, capturing the shape changes of dynamic Gaussians.

If the Gaussian shape remains unchanged during the dynamic process, we can directly use local_rotation to adjust the rotation of the Gaussians. However, in most scenes, Gaussians need to be reshaped across frames, with varying rotations and scales. Therefore, we use an independent d_rotation to modify the rotation. This adjustment can be applied either by adding to the quaternion or by multiplying it as a matrix.

[Greenpengrk]

Thanks for the quick reply! It is very helpful！