search

Angtian / NeuralVS

The Official Implementation of Neural View Synthesis and Matching for Semi-Supervised Few-Shot Learning of 3D Pose [NIPS 2021](https://arxiv.org/abs/2110.14213).
MIT License
20 stars 3 forks source link

Question about camera coordinate #1

Closed mlnyang closed 2 years ago

mlnyang commented 2 years ago

Hi Angtian, Thanks a lot for sharing your great works!

I have a question about your code in ProcessCameraParameters.py - get_transformation_matrix function. I believe that transformation matrix can be compute as [R^T | - R^T@C] (C is camera position in world coordinate) and the function above seems like acting same way. But i'm confused about why only y coordinate of camera is reversed,

136] C[0] = distance math.cos(elevation) math.sin(azimuth) 137] C[1] = -distance math.cos(elevation) math.cos(azimuth) 138] C[2] = distance * math.sin(elevation)

Is it the coordinate change for the openGL rendering?

Angtian commented 2 years ago

Thanks for your comment! Yes, it is equivalent to R^T @ C. The code uses the coordinate defined by the PASCAL3D+ dataset (I think it is the same as OpenGL). When I write the code, I refer to the project_3d.m function provided by the PASCAL3D+ dataset, which is similar to this code (https://github.com/yuxng/ObjectNet3D_toolbox/blob/master/project_3d.m).

mlnyang commented 2 years ago

Oh I got it now. Thanks!! :)
This may be a silly question, can i have one more?

The function get_transformation_matrix() turns the azimuth(z axis) and elevation(x axis) into rotation matrix So i thought that y axis will be orthogonal to image plane, at the generated camera space. Which will be the camera direction. But when I check the translation vector (- R^T@C), from pascal VOC dataset, the direction was same for all instances, toward z axis like(0,0,5).

At first thought, I believed that the translation vector will become like distance between camera and mesh vertices, but the camera direction(y axis) and translation vector direction(z axis) were different.. so i'm confused about them.

Angtian commented 2 years ago

Normally the annotation regarding object 6D pose will be R@X+T, where the T is the translation in camera coordinate. But in PASCAL3D+ they use a strange way to control the horizontal translation in 2D, they change the camera principle, which I think could have the same effects.

mlnyang commented 2 years ago

Okay I got it! thanks for your reply.