Add support for Air 2 and Air 2 Pro

[CGamesPlay]

I added the necessary code to detect and connect to the Air 2 and Air 2 Pro headsets (I didn't personally test but I had some of my users test it).

Since I don't have the hardware, I wanted to try to utilize the on-board config JSON to calculate the lens parameters. I know you said in Discord that you found them pretty inaccurate, but I did a comparison using the hand-calibrated values and what's on my device, and I found the results imperceptibly different. A few things to note:

• I had to customize the projection matrix based on the K matrix included. The main issue here is the off-center principal point in the matrix, which is effectively a translation of the lens.
• The on-board config data is weirdly translated, putting the right eye near x = 0. I compensate for the translation in the driver by centering them on the origin. It would probably be better to also rotate the values so that they fall on the x axis, but the difference is small.
• There's also a lens distortion array in the config JSON. The format is straightforward, it's an 18x32 list of (sx, sy, dx, dy) tuples, where sx, sy is a point on the device's screen and dx, dy is the linear distortion at that point. I haven't attempted to use this, so this is just a point of interest for future work.

[badicsalex]

Do I understand correctly that the newly introduced display_matrices function should probably replace imu_to_display_matrix?

I wonder how these calibration values interact with the actual IPD of the user.

[CGamesPlay]

Yes, it replaces imu_to_display_matrix (which returns the extrinsic matrix of the camera) and display_fov, which is derived from the intrinsic matrix of the camera (yfov = atan(h / 2 / fy)). For reference, here is the code my Mac app uses to create the projection matrix from the intrinsic matrix:

struct IntrinsicMatrix {
    /// Horizontal focal length, in pixels
    var fx: CGFloat
    /// Vertical focal length, in pixels
    var fy: CGFloat
    /// Principal point X coordinate
    var cx: CGFloat
    /// Principal point Y coordinate
    var cy: CGFloat
    /// Image width
    var w: CGFloat
    /// Image height
    var h: CGFloat

    init(_ from: DisplayMatrices) {
        guard from.intrinsic_matrix.len() == 9 else { fatalError("invalid intrinsic matrix") }
        fx = from.intrinsic_matrix[0]
        fy = from.intrinsic_matrix[4]
        cx = from.intrinsic_matrix[6]
        cy = from.intrinsic_matrix[7]
        w = CGFloat(from.resolution.w)
        h = CGFloat(from.resolution.h)
    }

    func projectionTransform(zNear: CGFloat, zFar: CGFloat) -> SCNMatrix4 {
        let a11 = 2 * fx / w
        let a22 = 2 * fy / h
        let a31 = 2 * cx / w - 1.0
        let a32 = 2 * cy / h - 1.0
        let a33 = -(zFar + zNear) / (zFar - zNear)
        let a43 = -2 * zFar * zNear / (zFar - zNear)

        return SCNMatrix4(
            m11: a11, m12: 0.0, m13: 0.0, m14: 0.0,
            m21: 0.0, m22: a22, m23: 0.0, m24: 0.0,
            m31: a31, m32: a32, m33: a33, m34: -1,
            m41: 0.0, m42: 0.0, m43: a43, m44: 0.0
        )
    }
}

Note that the projection matrix is transposed from how OpenGL does this. I have no idea why SceneKit wants this, but when you implement this in another framework you'll want to transpose this.

[badicsalex]

Ok, this seems like a well thought-out change then. I'll test it out in detail when I have some time and maybe deprecate the other two functions.

I will have to think about (and experiment with) IPD differences though. Do you do anything with that at all? Or are these calibration values calibrated to a specific 'median' IPD?

[CGamesPlay]

The translation component is likely calibrated to the focal point of the lenses (which I read from an XREAL forum post is 65mm apart). A human with a different IPD would experience some distortion from not having their eyes in the place the lenses expect. That said, it’s a pretty minor distortion, and the translation component can be outright ignored, which I believe would only affect the scale factor between real-world and virtual-world coordinates.

[CGamesPlay]

Do I understand correctly that the newly introduced display_matrices function should probably replace imu_to_display_matrix?

Also, we could view display_matrices as a more detailed version of these interfaces. I could write code that derives the data for imu_to_display_matrix and display_fov from these values and submit that as a PR, if you'd like.

[badicsalex]

I was planning to do exactly that, but a PR would also be helpful :D