search

badicsalex / ar-drivers-rs

Opensource Rust driver for various AR glasses
MIT License
135 stars 21 forks source link

compass/magnetometer missing for xreal light #15

Open roikr opened 5 months ago

roikr commented 5 months ago

nreal_light.rs does not return the magnetometer packet.

I'd parsed the not handled packet (type:'5', cmd id:'M') and the data is human readable (x10754y-5900z-6173) but I think I need some multiplier/divisor for it to make sense. did you managed to understand that message while reverse engineer the MCU firmware ? can you point me some direction ?

p.s. also there is unhandled packet ('A','K').

badicsalex commented 5 months ago

Hi!

I never bothered with the magnetometer, because it was borderline unusable for my purposes. (the Earth's magnetic field is so weak that the metal in the headrest of my chair could cause the compass to completely flip). I don't know what its units are, probably the magnetometer chip's (AK8975) raw values. You're welcome to parse the event though :)

The "A" packet is sent when the glasses goes to sleep, which is not very useful, and "K" is actually handled: https://github.com/badicsalex/ar-drivers-rs/blob/master/src/nreal_light.rs#L225-L234

roikr commented 4 months ago

the magnetometer can help with the yaw information when there are no strong magnetic disturbances. for 3 DOF you can't get nice results without visual tracker. maybe the message is sent by the mpu. however I am not familiar with the reverse engineering techniques you used with the MPU. but I red your post and you mention there are about 100 commands. can you point me some directions in the investigation of nreal light's MPU software and protocols ?

badicsalex commented 4 months ago

@roikr

As you've discovered, the magnetometer info comes from the MCU channel (and not the Ov580 like the IMU information). You can enable the magnetometer stream with the "1";"U" command, if disabled. All that needs to be done is it should be parsed, no need to do any more reverse engineering than what you've already did :)

P.S.: In a proprietary projet, I was able to get a pretty stable 3DOF without magnetometer, so it's definitely doable without a magnetometer.

roikr commented 4 months ago

reading the MCU magnetometer event, return this text message: 'x10436y-5644z-5651' the values are responsive for rotation of the glasses however they don't make sense. the norm of the vector should be fix for example. so I assume there are some missing scaling/calibration factors for each component. maybe they are the raw values of the AK8975 readings.

badicsalex commented 4 months ago

Oh yeah it's definitely uncalibrated, so the norm will definitely not be the same due to uncorrected soft and hard iron effects. You can actually calibrate the values by rotating the glasses all over trying to achieve minimum and maximum values of each coordinate, and you can calculate the offset = (max - min)/2 per coordinate. After applying that, the values will be relatively usable.

roikr commented 4 months ago

actually the reading from the magnetometer are pretty good but the main issue is the magnet of the clip-on frame (which I use as a reading glasses replacment) which add hard iron offset so every time I remove it, I need to recalibrate it. but if the clip-pn frame is fixed it is usable. I want to believe they solved it in air/ultra versions.

how ever it is really interesting how you got stable 3 DOF with gyro and accelerometer only (as you don't have any yaw reference) unless you had used the stereo cameras.

badicsalex commented 4 months ago

It's not stable, it's "pretty stable" :D Meaning it does have a few degrees of yaw every minute.

The trick is that you can actually calculate the yaw bias from the gravity vector (and the expectation that it continues to point down, not to the side) if the user turns their head up or down for a short while.

roikr commented 3 months ago

Hi, what about the timestamp for the magnetometer ? for some reason you left a comment in the deserialization, but no implementation.

badicsalex commented 3 months ago

@roikr

The timestamp was not needed for anything at the time, but for the magnetometer it could probably be parsed. I'm not sure how accurately that tracks the gyro/acc timestamp though, because that's literally a different chip, so some adjustments or faking (using the last acc/gyro timestamp as the magnetometer timestamp) might be necessary.

roikr commented 3 months ago

I am working on some ahrs that fuses gyro, acc and mag (complementary / EKF) I tried to use last acc timestamp for the magnetometer, but I still want to use to attached timestamp (hopefully the mcu and the imu share a clock. even if there is some small latency in the arrival the fusion may be more accurate. do you know the mcu timestamp type/representation ? I'v also notice that there are both gyro and acc timestamp but you've used only the gyro's. why is that ?

badicsalex commented 3 months ago

do you know the mcu timestamp type/representation ?

Unfortunately no, but I assume it's relatively easy to figure out.

you've used only the gyro's. why is that ?

The two are always the same. At least they looked the same to me.