Redefined K and R

[burritobrittany]

While constructing this toolbox I realized that while our toolboxes work, they are not entirely congruent with what we teach in the Bootcamp curriculum. Particularly our K and Rotation Matrices R do not match existing literature even though our code references them (Wolf 1983). Below explains the deviations:

We explain the pinhole camera projection model as the following: Note which direction Xc and Yc are pointing.

Then, we say we need to transform y and x to U and V by essentially incorporating a rotation and translation in our K matrix.

Most literature does not incorporate the rotation in their K matrix (-fy is not negative) but rather in the R matrix. Personally, I prefer it in the K matrix and found others who do too and refer to it as the affine matrix (http://www.cse.psu.edu/~rtc12/CSE486/lecture13.pdf). I think it makes logical sense and its easier to teach; it is part of the transformation from camera plane to UV coordinates. HOWEVER, look at our x and y axes and UV. Adding a minus to fy only rotates the y axis. The x axis is still reading right to left and not right to left. So our K matrix is technically incorrect and not really a rotation. The fx really should be negative too.

So why does our R matrix work? Well…our R matrix is not really a rotation either. If you look at angles2R in the original CIRN code or now intrinsicsExtrinsics2P and enter the values 0,0,0 for yaw, pitch, and roll (azimuth, tilt, swing) it provides a R of [1,0,0 ; 0 -1 0; 0 0 1]. This just flips the Y axis, not a true rotation and no longer a right hand system. It is altered to accommodate the sign switch in K.

So if it works…who cares?

I CARE DARNNIT!

Well, 1) I think we should teach the right thing. 2) I found this issue because I was trying to reverse engineer the code to work with rotation matrices provided by photoscan, etc. No matter what convention we will choose…there will be some further rotations and finagling. However as of right now…no correct rotation matrix will work with our definition of K.

Will our average bootcamper notice this…probably not. However, I think it would help people who are making their own tools and would like to use the basic building blocks of code…particularly for UAS applications whose output is in omega/kappa/phi or roll/pitch/yaw typically. It may be worth discussing if we want R formulation to be separate from intrinsicsExtrinsics2P like it was originally. People could throw in their own version of R instead of having to convert it to angles and then back to R.

Below I define the new rotation matrices if we were to do a CORRECT rotation in K, with –fx and -fy. I then implemented this in my code in intrinsicsExtrinsics2P. It should be congruent with our bootcamp slides except for the new negative sign for fx in K. The code still works with the same conventions of azimuth, swing, and tilt we always used. Also, we can change the definition of R matrix in our sides....it never matched the code anyway...so now it can!

Defining and Constructing the Camera Rotation Matrices using CIRN Defined Axes and K Matrix CIRN defines its Camera Axes (Subscript C) as the following:

So to construct the R that we use R_(W→C), we need to rotate the World Axis to Match the Camera Matrix.

The way the tool box defines its angles are Azimuth, Tilt, and Swing (A,T,S). At [A,T,S]=[0,0,0] the camera is pointing down with its Y_C axis aligned with YW. We will keep these definitions; they are pretty standard. To construct R(W→C) we will do a ZXZ rotation with ATS angles respectively; matching world to Camera axes.

Below shows the camera axes as if it were looking down (T=0) with a positive Azimuth A (Defined as positive CCW about X_C or in other words positive CW deviation of Y axes). It shows the following rotation to get the World X axes to match the Camera X axes. Using fact that cos(180-X)=-cos(x) and sin(180-X)=sin(x).

Positive tilt is defined as clockwise rotation about Xc or CW Deviation of Yc from the horizontal. (+90 degrees would be the camera looking out to North if A=0). Below shows the rotation to get the World Y and Z Axes to match the camera Axis Y and Z axes.

Positive Swing is defined as clockwise rotation about Zc. If you imagine that the camera is up looking at the horizon (Tilt 90); this would be if the camera was rotated counterclockwise looking from the back of the camera.

We then multiply the matrices. Remember, the order of rotation goes right to left!

So this R_(W→C) takes world coordinates (XYZ, and transforms them to Camera Coordinates XcYc,Zc). The K matrix will convert them from XcYcZc to UV. It assumes the K matrix has –fx and –fy.

[SRHarrison]

clap clap

why is your goPro backwards?

[burritobrittany]

clap clap

why is your goPro backwards?

hahaha I just couldnt get the axes to look right in the other direction, so I mirrored the image.

[KateBrodie]

@burritobrittany you rock. this has bothered me before but I couldn't figure out how to fix it. huge props to you!!! I will run the toolbox this weekend and make sure it sitll works and merge in your pull request.

[burritobrittany]

Hi all, @KateBrodie @mpalmsten

I made a couple of new changes to the code. They relate to the R change so it made sense to me to keep it in the same pull request.

1. I made a new function CIRNangles2R. This takes the pose angles as we define them and makes them into an R matrix to take World-->Camera (not UV but rather subscript C).

2. This slightly alters the intrinsicsExtrinsics2P function. However I think it makes it more versatile...if someone has different angle conventions (if imported from photoscan, etc) they can replace this module and just insert their own R.

3. Also, I changed the output of intrinsicsExtrinsics2P to also output the R, K, and IC matrices.

4. I did this because you can actually use the R*IC matrix to help eliminate bad that are not functional (outside of UV, or weird wrapping issues). Any point that gives a -Zc value is behind the plane of the image is not valid. I add this in the imageRectifier function where we also impose UV limits.

5. So I also removed some redundant code in intrinsicsExtrinsics2P. On the last line we normalize P by P./P(3,4). We dont need this for xyz2DistUV since we normalize by UV(:,3) to make homogenous. However we do for distUV2xyz...so i added it there. I am still a bit unclear on how that math works. However, the functions all run the same as before.

[KateBrodie]

@burritobrittany this ran successfully for me! I will approve and merge. Nice work!