rotation matrix convention seems inconsistent

[ykeuter]

🐛 Bugs / Unexpected behaviors:

quaternion_to_matrix seems to use a different convention (i.e. left-multiply) for the rotation matrix than Rotate expects as input during initialization (i.e. right-multiply). We could not find in the documentation if this is by design.

Instructions To Reproduce the Issue:

$ python3
Python 3.10.12 (main, Jul 29 2024, 16:56:48) [GCC 11.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import torch
>>> torch.__version__
'2.2.0+cu121'
>>> import pytorch3d
>>> pytorch3d.__version__
'0.7.7'
>>> from pytorch3d.transforms import Rotate, quaternion_to_matrix, quaternion_apply
>>> from math import sqrt
>>> q = torch.tensor([1. / sqrt(2.), 1. / sqrt(2.), .0, .0])
>>> p = torch.tensor([[.0, 1., .0]])
>>> quaternion_apply(q, p)
tensor([[0.0000, 0.0000, 1.0000]])
>>> Rotate(quaternion_to_matrix(q)).transform_points(p)
tensor([[ 0.,  0., -1.]])

We would expect the last two outputs to be identical, but the rotation seems to be inverted.

[bottler]

This is unfortunately expected behavior. Compare https://github.com/facebookresearch/pytorch3d/blob/main/pytorch3d/transforms/rotation_conversions.py#L18 and https://github.com/facebookresearch/pytorch3d/blob/main/pytorch3d/transforms/transform3d.py#L124 .