Transformation from NED to Cam

[amin-abouee]

Hi, Tartain air is my training dataset for a deep learning odometry project. In your dataset, the poses are in NED coordinate while I need to convert them to camera (opencv) coordinate system[right-down-forward]. For this purpose, you also implemented a function as ned2cam that converts a ned trajectory into my desired camera coordinate system. So the question is, why do you need to apply the right multiplication (.dot(T_inv)) for this transformation : ttt=T.dot(tt).dot(T_inv).

When you have the pose of a frame with respect to a world pose in NED coordinate, you just need to apply the transformation from NED to Cam (T matrix in your formula) to have it in Cam coordinate! So then, what is the point of T_inv there?

[ccj5351]

I think the code in ned2cam is correct.

In short:

• OpenCV style camera-to-world pose is calculated as:

$T^{w}_{c}$ = $T^{w}_{wned}$ * $T^{wned}_{cned}$* $T^{cned}_{c}$ =`T.dot(tt).dot(T_inv)` (as what you mentioned in the function `ned2cam`)

• note: $T^{w}_{wned}$ etc are in LaTex style if not shown correctly.

Assume we have the following coordinate systems:

• wned: world coordinate in NED (x Forward, y Right, z Down) format;
• cned: camera coordinate in NED (x Forward, y Right, z Down) format;
• w: world coordinate in OpenCV style (x Right, y Down, z Forward);
• c: camera coordinate in OpenCV style (x Right, y Down, z Forward);

Transformation from NED to OpenCV Style Coordinate System

Why do we need OpenCV style camera pose?

It is because we use the following pipeline to connect RGB, camera and world:

RGB image $(x,y)$ with $x$ to right, $y$ down, and image origin in the left-top corner ---> camera intrinsic K and inverse invK ---> camera points $P^{c}$ = $(X^{c}, Y^{c},Z^{c})$ ---> camera extrinsic E and inverse invE ---> world points $P^{w}$ = $(X^{w}, Y^{w},Z^{w})$.

How to get the transformation matrix from NED to OpenCV Style

• The matrix is defined as $T^{w}_{wned}$ to map the points $P^{wned}$ to the points $P^{w}$, i.e.,

$P^{w}$ = $T^{w}_{wned}$ * $P^{wned}$

• The matrix is also defined as $T^{c}_{cned}$ to map the points $P^{cned}$ to the points $P^{c}$, i.e.,

$P^{c}$ = $T^{c}_{cned}$ * $P^{cned}$

• To find $T^{w}_{wned}$ is to project (or to calculate the dot-product between) each axis (as a unit vector) of $x^{wned}$, $y^{wned}$, $z^{wned}$, into the axis $x^w$, $y^w$, $z^w$.

• To find $T^{w}_{wned}$ is to project (or to calculate dot-product between) each axis of $x^{wned}$, $y^{wned}$, $z^{wned}$, into the axis $x^w$, $y^w$, $z^w$. This is how the variable T is obtained.

• Do not forget that the transformation from NED to OpenCV, has to be applied twice: one for w and wned, another for c and cned. This is why you see 'dot(T_inv)' in the code.

[amin-abouee]

Thanks for your answer