Crazyflie 2.0 IMU driver reads absurd values

[KaranJain21]

Describe the bug I am using a crazyflie 2.0 with a fork of the firmware with the last PX4 commit coming from 5eddf32cd38f25e385a1a171319e4e08a3a4a4dc. On powering up the crazyflie, the IMU starts off by giving sensible readings (<0, 0, g> for the accelerometer, using a z-up coordinate frame), but after sometime, all the IMU readings become non-sensical. After some more time, it seems as though some of the IMU readings have switched places (e.g. gyro-y reading showing (a scaled version of) accelerometer-z). This jumbling makes me believe that the FIFO buffer is not being cleared properly and has some spurious bits left over (which pile up eventually causing some weird jumbling).

To Reproduce Steps to reproduce the behavior:

1. Flash the code to a crazyflie 2.0 and power it on
2. Read the IMU values (I have a thread subscribing to the published IMU uORB topics which I then can then print using the print_status of the thread). I have confirmed the spurious readings indeed happen at the lowest level by checking the accelerometer values at the driver level (MPU9250_I2C.cpp).
3. Wait for some time (unfortunately, this time is not repeatable and has ranged between a few seconds to almost 5 minutes at times)
4. The accelerometer and rate-gyro readings should now be non -sensical (sensible values for a static IMU -- <0,0,g> and <0,0,0>).

Expected behavior The IMU should consistently give sensible readings

Log Files and Screenshots Readings a few seconds after booting up (sensible)

Readings about 2 minutes after boot up (absurd)

[dagar]

I don't suppose you have a PX4 log (SD adapter or log streaming)?

Once the mpu9250 gets into this state do you know if restarting the driver produces valid data again or does the entire system need a power cycle?

After some more time, it seems as though some of the IMU readings have switched places (e.g. gyro-y reading showing (a scaled version of) accelerometer-z).

This part sounds like things have gotten out of sync with the FIFO.

[KaranJain21]

Sorry, I do not have the PX4 log.

Just did a test and restarting the driver starts producing valid data again. However, a similar behavior is again observed some time after restarting the driver.

It's probably also worth mentioning that I am running the IMU at 500Hz by changing line 76 in MPU9250_I2C.hpp to: static constexpr float FIFO_SAMPLE_DT{1e6f / 500.f};. This originally had a value of FIFO_SAMPLE_DT{1e6f / 1000.f}

I did try running the IMU at a 1000Hz too, and at that rate the non-sensical data behavior is observed sooner (usually within 30 seconds, but again not really repeatable).

[dagar]

It's probably also worth mentioning that I am running the IMU at 500Hz by changing line 76 in MPU9250_I2C.hpp to: static constexpr float FIFO_SAMPLE_DT{1e6f / 500.f};. This originally had a value of FIFO_SAMPLE_DT{1e6f / 1000.f}

I did try running the IMU at a 1000Hz too, and at that rate the non-sensical data behavior is observed sooner (usually within 30 seconds, but again not really repeatable).

That's an important detail, you need to make sure FIFO_SAMPLE_DT matches how the sensor is actually configured. https://github.com/PX4/PX4-Autopilot/blob/1317b1a6e14e3178c5e5360dc309e3903b7d1324/src/drivers/imu/invensense/mpu9250/MPU9250_I2C.hpp#L155-L166

The FIFO is still filling at the same rate, but now the driver isn't running fast enough.

Can you try setting the sample rate divider as well?

[KaranJain21]

I am not sure which variable to change to set the sample rate divider. Could you please help me with that?

I tried modifying the SAMPLES_PER_TRANSFER variable to 2 instead of 1, but the same behavior persists, so this is probably not the one I want to change.

[KaranJain21]

Regardless of that, the IMU driver not functioning as expected even when I'm running it at 1000Hz, means that the cause of the underlying behavior is probably something different.

@dagar do you have any leads as to what more I can check?