About isotropic and anisotropic Metrics in Experiments.

[Codefmeister]

Hi, yewzijian! Thanks for your excellent work and compact declaration. It really inspires me a lot. But I got a little confusion in Experiments Part. In paper, you mentioned that the euler angle error and translation error were anisotropic, while another one was isotropic. Sorry that I cannot understand anisotropy in metric very well. Could you plz give a more specific explanation? Thank you in advance.

[yewzijian]

Hi, I should have defined them more closely in the paper, but..

In short, anisotropic metrics are dependent on the direction of the error, so might be biased to certain error directions. Particularly, the original metrics in DCP aren't very typical for measuring rotation error, so I replaced it with a more common measure.

Say the groundtruth is at identity rotation (0°, 0°, 0°), and that our prediction is (0°, 30°, 30°) in terms of Euler angles. The MAE measure in DCP (which I term as anisotropic) will compute the error as:

MAE = |0 - 0| + |0 - 30| + |0 - 30| = 60°

where |x| denotes the absolute value of x. However, notice that in its rotation matrix form, the predicted rotation is:

[[ 0.866 -0.     0.5  ]
 [ 0.25   0.866 -0.433]                       (*)
 [-0.433  0.5    0.75 ]]

which is a 42° rotation around the axis [0.6947 0.6947 0.1862]. As such, the metric unfairly penalizes this rotation and rate it worse than say a predicted rotation of [0, 0, 50°] even though it is actually closer to the groundtruth

Furthermore, the use of Euler angles makes the metric dependent on which axis convention we use. DCP uses the convention 'zyx', but if we instead use a 'xyz' convention, the same rotation () will be (33.7°, 25.7°, 16.1°) in Euler angles which will give you a MAE of 75.3°. Thus, the more typical metric for rotation error is ∠(R' R^T), where ∠(.) denotes the computation of the angular magnitude of the rotation matrix. This is the computation used for my isotropic measures.

For the translation metrics, I considered the translation portion of the pose error T_gt^-1 * T_pred which arguably is a better measure than simply comparing the raw translation components.

Both DCP's and my metrics are computed in my code, which you can refer to for further details.

Regards, Zi Jian

[Codefmeister]

Appreciate for your patient and clear answer. Your choices of metric are right!