search

badicsalex / ar-drivers-rs

Opensource Rust driver for various AR glasses
MIT License
124 stars 17 forks source link

compass/magnetometer missing for xreal light #15

Open roikr opened 1 month ago

roikr commented 1 month ago

nreal_light.rs does not return the magnetometer packet.

I'd parsed the not handled packet (type:'5', cmd id:'M') and the data is human readable (x10754y-5900z-6173) but I think I need some multiplier/divisor for it to make sense. did you managed to understand that message while reverse engineer the MCU firmware ? can you point me some direction ?

p.s. also there is unhandled packet ('A','K').

badicsalex commented 4 weeks ago

Hi!

I never bothered with the magnetometer, because it was borderline unusable for my purposes. (the Earth's magnetic field is so weak that the metal in the headrest of my chair could cause the compass to completely flip). I don't know what its units are, probably the magnetometer chip's (AK8975) raw values. You're welcome to parse the event though :)

The "A" packet is sent when the glasses goes to sleep, which is not very useful, and "K" is actually handled: https://github.com/badicsalex/ar-drivers-rs/blob/master/src/nreal_light.rs#L225-L234

roikr commented 2 weeks ago

the magnetometer can help with the yaw information when there are no strong magnetic disturbances. for 3 DOF you can't get nice results without visual tracker. maybe the message is sent by the mpu. however I am not familiar with the reverse engineering techniques you used with the MPU. but I red your post and you mention there are about 100 commands. can you point me some directions in the investigation of nreal light's MPU software and protocols ?

badicsalex commented 2 weeks ago

@roikr

As you've discovered, the magnetometer info comes from the MCU channel (and not the Ov580 like the IMU information). You can enable the magnetometer stream with the "1";"U" command, if disabled. All that needs to be done is it should be parsed, no need to do any more reverse engineering than what you've already did :)

P.S.: In a proprietary projet, I was able to get a pretty stable 3DOF without magnetometer, so it's definitely doable without a magnetometer.

roikr commented 2 weeks ago

reading the MCU magnetometer event, return this text message: 'x10436y-5644z-5651' the values are responsive for rotation of the glasses however they don't make sense. the norm of the vector should be fix for example. so I assume there are some missing scaling/calibration factors for each component. maybe they are the raw values of the AK8975 readings.

badicsalex commented 2 weeks ago

Oh yeah it's definitely uncalibrated, so the norm will definitely not be the same due to uncorrected soft and hard iron effects. You can actually calibrate the values by rotating the glasses all over trying to achieve minimum and maximum values of each coordinate, and you can calculate the offset = (max - min)/2 per coordinate. After applying that, the values will be relatively usable.

roikr commented 3 days ago

actually the reading from the magnetometer are pretty good but the main issue is the magnet of the clip-on frame (which I use as a reading glasses replacment) which add hard iron offset so every time I remove it, I need to recalibrate it. but if the clip-pn frame is fixed it is usable. I want to believe they solved it in air/ultra versions.

how ever it is really interesting how you got stable 3 DOF with gyro and accelerometer only (as you don't have any yaw reference) unless you had used the stereo cameras.

badicsalex commented 3 days ago

It's not stable, it's "pretty stable" :D Meaning it does have a few degrees of yaw every minute.

The trick is that you can actually calculate the yaw bias from the gravity vector (and the expectation that it continues to point down, not to the side) if the user turns their head up or down for a short while.