Implementation issue in gradient (surface normal) computation

[LifeBeyondExpectations]

Thanks for sharing the code. I wonder whether this code is correct or not.

torch.einsum('bi,bij->bj', gradients, grads_inv)

which is located at https://github.com/MoyGcc/vid2avatar/blob/4c7e640200151e2210077459faac3e85e6b8a086/code/lib/model/v2a.py#L282

---

Let me describe my understanding. The variable gradients indicates the surface normal vector at the canonical space,

gradients = torch.autograd.grad(
            outputs=sdf, # signed distance function
            inputs=pnts_c, # points at the canonical space
            grad_outputs=d_output,
            create_graph=create_graph,
            retain_graph=retain_graph,
            only_inputs=True,
)[0]

which is located at https://github.com/MoyGcc/vid2avatar/blob/4c7e640200151e2210077459faac3e85e6b8a086/code/lib/model/v2a.py#L274

Accordingly, to compute the surface normal vector at the deformed space, we need to use the Jacobian matrix (pnts_d over pnts_c) which is identical to the variable grads below,

grads = []
        for i in range(num_dim):
                d_out = torch.zeros_like(pnts_d, requires_grad=False, device=pnts_d.device)
                d_out[:, i] = 1
                grad = torch.autograd.grad(
                        outputs=pnts_d,
                        inputs=pnts_c,
                        grad_outputs=d_out,
                        create_graph=create_graph,
                        retain_graph=True if i < num_dim - 1 else retain_graph,
                        only_inputs=True,
                )[0]
                grads.append(grad)
grads = torch.stack(grads, dim=-2)
grads_inv = grads.inverse()

Based on these variables, I could compute the surface normal at the deformed space as

gradients_at_deformed_space = torch.bmm(grads, gradients.reshape(-1, 3, 1)).reshape(-1,3)

which is identical to the code below.

torch.einsum('bi,bij->bj', gradients, grads)

TL;DR) grads_inv -> grads

Can you check this issue? If I am wrong, please let me know.

Thanks.

[MoyGcc]

Hi, thanks for your interest and the detailed explaination. I think the disagreement here is the Jacobian matrix. In my understanding, it should be pnts_c over pnts_d which you can refer to Eq. 6 in the paper.

[LifeBeyondExpectations]

Got it. My mistake. Thanks for your comments and let me close this issue.