bones_dfm or bones_rst when using gauss_mlp_skinning?

[Lxiangyue]

Hi, many thanks for your excellent work!

May I ask (1) why using 'bones_dfm' instead of 'bones_rst' in the below line? Whether (2) the 'skin_backward' is the LBS weight applied to sample pts in the camera coordinate and transfer them to the root coordinate? and (3) the 'xyz_coarse_sampled' is the sample pts in the root coordinate? If so, it seems like using 'bones_rst' here makes sense.

https://github.com/facebookresearch/banmo/blob/177a39041f5464f2e45b505ac0fc04ddc620b60e/nnutils/rendering.py#L291

[gengshan-y]

Hi, here's some relations of transformations that may help clarify things.

Backward:

X(view space at t) -> root pose inverse -> X(root space at t) -> backward LBS ->  X(root space at rest pose)

Forward:

 X(root space at rest pose) -> forward LBS -> X(root space at t') -> root pose -> X(view space at t') 

We calculate backward skinning weights with time t pose, and forward skinning weights with rest pose, and both are in the root space. In that line of code, xyz_coarse_sampled are points in the root space at time t. Let me know if there are further questions.

[Lxiangyue]

Many thanks for your clear and rapid reply!

Ah, I got it! So at the notation (# backward skinning), the 'xyz_coarse_sampled' into the lbs() is X(root space at t), and the returned 'xyz_coarse_sampled' is X(root space at rest pose). That's clear now. https://github.com/facebookresearch/banmo/blob/177a39041f5464f2e45b505ac0fc04ddc620b60e/nnutils/rendering.py#L293