Implement Kalman filter to read Quaterion or Euler Angles from continous ARGlasses connection.

[tribbloid]

I'd like to introduce capabilities similar to the Windows counterpart of this project:

https://github.com/MSmithDev/AirAPI_Windows

Which defined 2 API functions:

float* GetQuaternion()
{
    mtx.lock();
    q[0] = qt.array[0];
    q[1] = qt.array[1];
    q[2] = qt.array[2];
    q[3] = qt.array[3];
    mtx.unlock();
    return q;
}

float* GetEuler()
{

    mtx.lock();
    e[0] = euler.angle.pitch;
    e[1] = euler.angle.roll;
    e[2] = euler.angle.yaw;
    mtx.unlock();
    return e;
}

The returned data could be easily updated by fusing 3-axis compass & 3-axis gyroscope with EKF/UKF. This capability can be enabled for all glasses.

Would you like to review it once it is finished?

I'm uncertain about your plan for the following code:

    fn display_matrices(&self) -> Result<(DisplayMatrices, DisplayMatrices)> {
        Err(Error::NotImplemented)
    }

is it supposed to be the rotation orthogonal matrix for IMU-only glasses and transformation matrix for tracked glasses? If so, we don't need to implement it immediately.

[badicsalex]

Hi!

Sure, it would be great if someone implemented (and maintained :) ) fusion! I'd gladly help review or maybe even test it. Do you want to contribute it to this crate or a separate one?

display_matrices is going to be the "the API of choice" to get the transformation from the IMU frame to the display. It is a more general version of display_fov + imu_to_display_matrix (which are going to be deprecated), but I did not have time to convert the data from existing glasses to the new method.

[tribbloid]

thanks a lot! I obviously prefer PR to the same crate if it is not published

I also can't find implementation of imu_to_display_matrix that uses GlassesEvent, also no example was written for them.

Are they useful to the sensor fusion?

[badicsalex]

Only read_event is needed for fusion, the rest is for display (and some other stuff)

[tribbloid]

first draft is out, I only have XReal Air 1 which has no magnetometer, so I may need others to test for all sensors.

wondering if the format & extra maintenance work (toolchain, format) looks already to you

[tribbloid]

I have 2 questions:

• is there a way to destroy an ARGlasses instance by cutting off connection?
• what will happen to GlassesEvent that are not polled in time? Will they be buffered or simply discarded?

[badicsalex]

Thanks @tribbloid , I never realized it would be this easy to get started with a Fusion implementation, it's definitely great to have this in the crate. I made a few review comments in #11

To answer your questions:

1. If you drop the ARGlasses object, the connections should close. If that does not happen, that should be reported as a bug.
2. They get discarded on most implementations IIRC.

[tribbloid]

just got my glasses (XReal Air 1) back, turns out a lot of my assumptions are wrong.

most importantly, rust quaternion library always converts to/from Euler angles of FRD reference frame (by axis, roll, pitch, yaw), but GlassEvent uses RUB frame for both position & rotation.

I guess it's too late to revise GlassEvent convention, but would you like the Fusion to also represent Euler angles in RUB frame, both internally & externally?

[badicsalex]

I wouldn't use Euler angles at all, to be honest (at least in this crate). If someone needs those angles, they can call euler_angles on the returned Quaternion object themselves.

BTW, in the eskf fusion code, instead of UnitQuaternion::from_euler_angles(d_S1_t1.y, d_S1_t1.x, d_S1_t1.z); they actually use UnitQuaternion::from_scaled_axis(d_S1_t1);, which automatically conforms to the chosen coordinate system, and is actually more precise.