WaldJohannaU / RIO10

22 stars 6 forks source link

Theoretical Issue with Evaluation script #8

Closed Shubodh closed 1 year ago

Shubodh commented 2 years ago

In RIO10 paper, for evaluating how close predicted pose is to ground truth pose, the Quaternion Angular Error is defined as

|| 180/pi 2 arccos(dot(q_pred_inv, q_gt)) ||

where q_pred_inv is inverse of q_predicted and q_gt is ground truth, dot means dot product.

However in your code, it is different:

The Quaternion Angular Error is defined here in your code. You didn't do inverse for quaternion here, i.e. it is

|| 180/pi 2 arccos(dot(q_pred, q_gt)) ||

instead of

|| 180/pi 2 arccos(dot(q_pred_inv, q_gt)) ||.

Basically inverse of q_pred is missing.

Am I missing something here? Or is your code incorrect?

(Please excuse the unneccessary title edits)

Shubodh commented 2 years ago

I relooked again at this and I think there is no issue with your code, even though it doesn't explicitly do inverse, it looks correct. I am explaining below:

In my original comment, I wrote that the Quaternion Angular Error is defined in your paper as

|| 180/pi 2 arccos(dot(q_pred_inv, q_gt)) ||

But actually it is

|| 180/pi 2 arccos(product(q_pred_inv, q_gt)) ||.

"dot" and "product" would mean two different things. For example, if say there are 2 (same) quaternions X and Y which are (a + bi + cj + dk) and (a + bi + cj + dk), then "dot" of these two as 4-vectors [a,b,c,d] & [a,b,c,d] would be (a^2 + b^2 + c^2 + d^2) whereas "product" of the two quaternions would be (a^2 - b^2 - c^2 - d^2 + more terms like abi + aci +...) .

Therefore, even though it is written as quat_inv in the paper, what is implemented in code is correct. Basically,

|| 180/pi 2 arccos(product(q_pred_inv, q_gt)) ||.

is same as

|| 180/pi 2 arccos(dot(q_pred, q_gt)) ||. Note two differences: (product vs dot) and (q_pred_inv vs q_pred).

So if we continue the X and Y example above. X = (a + bi + cj + dk) and Y = (a + bi + cj + dk) (Assuming unit quaternions)

product(X_inv, Y) = (a^2 + b^2 + c^2 + d^2) (This is what paper says)

which is the same as

dot(X, Y) = dot([a,b,c,d],[a,b,c,d]) = (a^2 + b^2 + c^2 + d^2). (This is what is implemented in code)

So what is mentioned is paper seems to be same as that in code mathematically.