Wrong camera projection matrix

[osrf-migration]

Original report (archived issue) by Christian Rauch (Bitbucket: christian-rauch).

---

Sensor definition

I have a camera sensor with intrinsic parameters:

#!XML

        <camera>
          <!-- FOV = 2 * arctan(w/(2*f_x)), with f_x = 536.473145060694 -->
          <horizontal_fov>1.0756668597435934</horizontal_fov>
          <image>
            <format>R8G8B8</format>
            <width>640</width>
            <height>480</height>
          </image>
          <clip>
            <near>0.0001</near>
            <far>1000</far>
          </clip>
          <distortion>
            <k1>0.03310900261913022</k1>
            <k2>-0.1081296635444241</k2>
            <p1>0.0003885055533487762</p1>
            <p2>-0.001167055731123937</p2>
            <k3>0</k3>
            <!-- relative centre with c = (314.0086519801989, 246.7391576005324) -->
            <center>0.49063851871906083 0.5140399116677759</center>
          </distortion>
        </camera>

Expected projection matrix

According to http://www.songho.ca/opengl/gl_projectionmatrix.html#perspective, the projection matrix is: For the given camera parameters this should be something around:

P: 
   1.67648          0  -0.018723          0
         0   -2.23693 -0.0280798          0
         0          0          1    -0.0002
         0          0          1          0

The signs are different because this is in a RDF (x - Right, y - Down, z - Front) coordinate system.

Projection matrix used in camera sensor

But the projection matrix (Camera::ProjectionMatrix()) gives (row-major):

1.67647 0 0 0 0 2.2353 0 0 0 0 -1 -0.0002 0 0 -1 0

which is missing values for P(0,2) and P(1,2) (they are set to 0).

Since Gazebo 10 I can update the camera intrinsics via Camera::UpdateCameraIntrinsics:

#!c++

UpdateCameraIntrinsics(536.473145060694, 536.8639285109052, 314.0086519801989, 246.7391576005324, 1.0);

or compute the projection matrix manually via Camera::BuildProjectiveMatrix:

#!c++

BuildProjectiveMatrix(640, 480, 536.473145060694, 536.8639285109052, 314.0086519801989, 246.7391576005324, 1.0, 0.0001, 1000);

in which case I get:

1.67648 0.003125 0.018723 0 0 2.23693 -0.02808 0 0 0 -1 -0.0002 0 0 -1 0

as projection matrix. This is still not correct since this time P(0,1)=0.003125, which should be 0.

[osrf-migration]

Original comment by Christian Rauch (Bitbucket: christian-rauch).

---

• Edited issue description

[osrf-migration]

Original comment by Christian Rauch (Bitbucket: christian-rauch).

---

• Edited issue description

[osrf-migration]

Original comment by Christian Rauch (Bitbucket: christian-rauch).

---

I wrongly assumed that S is the scale, but it's the skew.

The correct call to UpdateCameraIntrinsics:

#!c++

UpdateCameraIntrinsics(536.473145060694, 536.8639285109052, 314.0086519801989, 246.7391576005324, 0);

sets the correct projection matrix: 1.67648 0 0.018723 0 0 2.23693 -0.02808 0 0 0 -1 -0.0002 0 0 -1 0

It's however still an issue that this projection matrix is not set correctly at the initial construction of the camera sensor.