Convert 3D coordinates to 2D

wematan commented 6 years ago

Is there an easy way to convert x3D y3D z3D to 2D x,y pixels? The output only contains 3D coordinates.

Thanks.

fabbrimatteo commented 6 years ago

Firstly, you have to perform a roto-translation to have the origin in correspondence to the camera view-point:

$\huge \begin{bmatrix} X'\\ Y'\\ Z' \end{bmatrix} = \left ( \begin{bmatrix} X\\ Y\\ Z \end{bmatrix} - \begin{bmatrix} X_{\mathrm{cam}}\\ Y_{\mathrm{cam}}\\ Z_{\mathrm{cam}} \end{bmatrix} \right )\cdot \begin{bmatrix} \cos\beta \cos\gamma & -\cos\alpha \sin\gamma + \sin\alpha \sin\beta \cos\gamma & \sin\alpha \sin\gamma + \cos\alpha \sin\beta \cos\gamma \\ \cos\beta\sin\gamma & \cos\alpha \cos\gamma + \sin\alpha \sin\beta \sin\gamma & - \sin\alpha \cos\gamma + \cos\alpha \sin\beta \sin\gamma \\ -\sin\beta & \sin\alpha \cos\beta & \cos\alpha \cos\beta \\ \end{bmatrix}$

Where:

[X, Y, Z] are the 3D coords that you want to convert
[Xcam, Ycam, Zcam] are the 3D coords of the camera
[α, β, γ] are the components of the camera rotation

After that, using the field of view parameter (that you can find in the annotation) you have to calculate the field of view:

$f = \frac{H}{2} \cdot \cot\left ({\frac{FOV}{2}} \right )$

Now, project the transformed 3D coords (X', Y', Z') over the image plane in order to find 2D coords x and y:

$\small \left\{\begin{matrix} x = & f \cdot \frac{X'}{Y'} + \frac{W}{2}\\ y = & \frac{H}{2} - f \cdot \frac{Z'}{Y'}\\ \end{matrix}\right.$

Where W and H are the width and height of the 2D plane (in our case are 1920 and 1080).

Note: all angles are expressed in radiants.

wematan commented 6 years ago

Hi Thanks for the info.

Few other questions regarding the conversion:

FOV is in Degrees? and if so do i need to convert it to radians in order to calculate 1/tan(Fov/2)?
do i need to convert α, β, γ to radians as well? Thanks.

fabbrimatteo commented 6 years ago

Yes, in our .csv, all the angles are in degrees. So, you have to convert FOV, α, β and γ to radiants.

mohammadakz commented 5 years ago

@fabbrimatteo Hi thanks for your explanation, what do you mean about "[α, β, γ] are the components of the camera rotation". Does it mean that the camera is rotated or not? If you could explain, please. thanks in advance

fabbrimatteo commented 5 years ago

The camera (and every object in a space) is always rotated to some quantity with respect to the reference system of the world. And you need thos values if you want to change your reference system according to the camera.

montmejat commented 2 years ago

Hello, sorry for bringing this back up and thanks for what you have shared. I'm using the method you have talked about here (_WORLD3D_TO_SCREEN2D has the same effect) but I'm getting some weird distortion effects around the edges. Do you know if there would be anyway to fix this? Thanks.

GTA Image Distortion

fabbrimatteo commented 2 years ago

Try using this mod: https://www.gta5-mods.com/misc/no-chromatic-aberration-lens-distortion-1-41

montmejat commented 2 years ago

That perfectly fixed my problem, thanks! 😄

mshooter commented 1 year ago

Firstly, you have to perform a roto-translation to have the origin in correspondence to the camera view-point:

Where:

[X, Y, Z] are the 3D coords that you want to convert

[_X_cam, _Y_cam, _Z_cam] are the 3D coords of the camera

[α, β, γ] are the components of the camera rotation

After that, using the field of view parameter (that you can find in the annotation) you have to calculate the field of view:

Now, project the transformed 3D coords (X', Y', Z') over the image plane in order to find 2D coords x and y:

Where W and H are the width and height of the 2D plane (in our case are 1920 and 1080).

Note: all angles are expressed in radiants.

Hi, a quick question. In which coordinate system was this? and what is the Up axis?

fabbrimatteo commented 1 year ago

I do not have such information anymore. But with the new annotations you can use this formula:

$\begin{cases} x = f \cdot \frac{X'}{Z'} + \frac{W}{2}\\ y = f \cdot \frac{Y'}{Z'} + \frac{H}{2}\\ \end{cases}$

Take a look at the pinhole camera model: https://docs.opencv.org/3.4/d9/d0c/group__calib3d.html

mshooter commented 1 year ago

Is the focal length in mm or in pixels?

fabbrimatteo commented 1 year ago

The focal length is in pixels.

fabbrimatteo / JTA-Mods

Convert 3D coordinates to 2D #3