I'm trying to differentiate through MANO layer with rotation matrices mode for both root joint and other joints. I have a question regarding backward pass through SVD in batch_rotprojs function. As we are dealing with rotation matrices, the singular values will always be 1. However, as mentioned in https://pytorch.org/docs/stable/generated/torch.svd.html, the gradient will only be finite when the input does not have zero nor repeated singular values, which definitely contradict with our case here. So I'm wondering whether there is a workaround for this, or I have to stick with axis angles which have no such problems.
p.s., I'm considering differentiating through rotation matrices instead of axis angles, since according to some resources e.g. https://arxiv.org/pdf/2003.09572.pdf, using trigonometric functions tend to be more difficult to train since they are non-injective. As far as I know, converting axis angles involve trigonometry so I decide to skip the process if possible. Indeed, I've also tried training to regress axis angles which does not seem to converge.
Any suggestion would be appreciated! Thanks in advance.
Thanks for your re-implementation of MANO layer.
I'm trying to differentiate through MANO layer with rotation matrices mode for both root joint and other joints. I have a question regarding backward pass through SVD in
batch_rotprojsfunction. As we are dealing with rotation matrices, the singular values will always be 1. However, as mentioned in https://pytorch.org/docs/stable/generated/torch.svd.html, the gradient will only be finite when the input does not have zero nor repeated singular values, which definitely contradict with our case here. So I'm wondering whether there is a workaround for this, or I have to stick with axis angles which have no such problems.p.s., I'm considering differentiating through rotation matrices instead of axis angles, since according to some resources e.g. https://arxiv.org/pdf/2003.09572.pdf, using trigonometric functions tend to be more difficult to train since they are non-injective. As far as I know, converting axis angles involve trigonometry so I decide to skip the process if possible. Indeed, I've also tried training to regress axis angles which does not seem to converge.
Any suggestion would be appreciated! Thanks in advance.