curobo.geom.transform.pose_inverse or Pose.inverse() has incorrect gradients for quaternion

[YuhengZhi]

If it’s not a bug, please use discussions: https://github.com/NVlabs/curobo/discussions

Please provide the below information in addition to your issue:

1. cuRobo installation mode (choose from [python, isaac sim, docker python, docker isaac sim]): isaac sim
2. python version: 3.10
3. Isaac Sim version (if using): 4.1.0

---

Issue Details

Minimal example: a pose with zero position and non-zero rotation is created and normalized. The loss function is defined as a selected dimension (default as 0 here) of the quaternion of the inverse of that pose. Also, I had to remove these two None's to make backward work: https://github.com/NVlabs/curobo/blob/a027cbcf380df07a10e7b956990f79240e951dc1/src/curobo/geom/transform.py#L1226-L1227

from curobo.types.math import Pose
from curobo.geom.transform import pose_inverse
import torch

position = torch.tensor([0.0, 0.0, 0.0], device="cuda")
quaternion = torch.tensor([1., 1., 1., 1.], device="cuda")
pose = Pose(
    position=position,
    quaternion=quaternion,
    normalize_rotation=True
)
position = pose.position.requires_grad_(True)
quaternion = pose.quaternion.requires_grad_(True)
print(f'position: {position}')
print(f'quaternion: {quaternion}')

def loss_f(position, quaternion, idx=0):
    inv_pos, inv_quat = pose_inverse(position, quaternion)
    return inv_quat[0, idx]  + inv_pos.mean()

position.grad = None
quaternion.grad = None
loss = loss_f(position, quaternion)
loss.backward()

print(f'position.grad: {position.grad}')
print(f'quaternion.grad: {quaternion.grad}')

success = torch.autograd.gradcheck(loss_f, inputs=(position, quaternion, 0), eps=1e-3)

Output:

position: tensor([[0., 0., 0.]], device='cuda:0', requires_grad=True)
quaternion: tensor([[0.5000, 0.5000, 0.5000, 0.5000]], device='cuda:0', requires_grad=True)
position.grad: tensor([[-0.3333, -0.3333, -0.3333]], device='cuda:0')
quaternion.grad: tensor([[0., 0., 0., 0.]], device='cuda:0')

Position's gradients are correct but the quaternion has zero gradient as long as position=[0, 0, 0]. Changing quaternion's value does not change its gradients.

[YuhengZhi]

Actually, I seem to have found a fix as in #393 .