search

PX4 / PX4-Autopilot

PX4 Autopilot Software
https://px4.io
BSD 3-Clause "New" or "Revised" License
8.56k stars 13.55k forks source link

lsm303d & l3gd20 can not calibrate level horizon #14505

Open rolandash opened 4 years ago

rolandash commented 4 years ago

Describe the bug lsm303d & l3gd20 can not be calibrate leveled in single calibration process. This issue should infect all FCUs have these 2 sensors onboard (>= v1.10). This issue can only be visible/detected when FCU has only these 2 sensors on-board (or uses these 2 as primary motion sensors? or equivalent to disable other sensor's driver?) This issue does not exist in earlier version of version (<=v1.9). This issue is not observed with other type of IMU sensors so far.

To Reproduce Steps to reproduce the behavior:

  1. Use a FCU that has and only has lsm303d & l3gd20 on board. MindPX users is more convenient as just need to disconnect the isolated redundant IMU module. If you are using Pixhawk, you may have to remove other sensors on-board (should be equivalent to disable drivers for other sensors onboard, but I did not test that case).
  2. connect to QGC.
  3. Clear/reset all parameters to start over.
  4. Select a generic airframe, like Generic Quad X.
  5. Perform a complete sensor calibration process, util 'level horizon' finished and all indicators becomes green.
  6. Place FCU in level, go to fly view and check the flight gadget. The attitude gadget is drifting and not leveled.

Expected behavior The attitude gadget should be leveled.

Additional context The issue can be corrected by a second 'level horizon' action.

  1. Following above step 6, do a 'level horizon' action again.
  2. reboot the FCU and connect to QGC.
  3. The attitude gadget is leveled this time.
dagar commented 4 years ago

Would you be able to get a log from boot (SDLOG_MODE 2) with an attempted calibration?

Capturing this output from the console might also help.

sensors status

listener sensor_accel

listener sensor_gyro

work_queue status

rolandash commented 4 years ago

@dagar log below as you requested.

Update I have tested that disabling other sensor's driver is equivalent to physically remove them from board. So if you got a Pixhawk, then you can reproduce it simply by disable other motion sensor's driver and only left lsm303d & l3gd20 there.

Below's log is recorded from a Pixhawk running pr-new_invensense_drivers branch, with mpu6k driver disabled.

FMUv2 ver 0x8 : Rev 0 V2
[boot] Fault Log info File No 4 Length 3177 flags:0x01 state:1
[boot] Fault Log is Armed
sercon: Registering CDC/ACM serial driver
sercon: Successfully registered the CDC/ACM serial driver
HW arch: PX4_FMU_V3
HW type: V2
HW version: 0x00090008
HW revision: 0x00000000
FW git-hash: 738c3939a796bd70216763246a6ec432ea7d4f90
FW version: 1.11.0 0 (17498112)
FW git-branch: pr-new_invensense_drivers
OS: NuttX
OS version: Release 8.2.0 (134349055)
OS git-hash: b902d4ca3d79ffbd5148670cb13bfeef7d917555
Build datetime: Mar 27 2020 18:05:56
Build uri: localhost
Toolchain: GNU GCC, 7.2.1 20170904 (release) [ARM/embedded-7-branch revision 255204]
PX4GUID: 000100000000373936353235510a004a0026
MCU: STM32F42x, rev. 3
nsh: mount: mount failed: No such device
nsh: mkfatfs: mkfatfs failed: No such device
INFO  [param] selected parameter default file /fs/mtd_params
INFO  [tune_control] Publishing standard tune 16
Board defaults: /etc/init.d/rc.board_defaults
WARN  [dataman] Could not open data manager file /fs/microsd/dataman
ERROR [dataman] dataman start failed
nsh: rgbled_pwm: command not found
Board sensors: /etc/init.d/rc.board_sensors
rgbled #0 on I2C bus 2
ms5611 #0 on SPI bus 1 (devid=0x3e)
l3gd20 #0 on SPI bus 1 (devid=0x22)
lsm303d #0 on SPI bus 1 (devid=0x11)
Board extras: /etc/init.d/rc.board_mavlink
INFO  [mavlink] mode: Config, data rate: 800000 B/s on /dev/ttyACM0 @ 57600B
Starting Main GPS on /dev/ttyS3
Starting MAVLink on /dev/ttyS1
INFO  [mavlink] mode: Normal, data rate: 1200 B/s on /dev/ttyS1 @ 57600B
ERROR [mavlink] DM_KEY_MISSION_STATE lock failed
ERROR [mavlink] offboard mission init failed (-1)
INFO  [px4io] default PWM output device
INFO  [init] Mixer: /etc/mixers/quad_x.main.mix on /dev/pwm_output0
INFO  [init] Mixer: /etc/mixers/pass.aux.mix on /dev/pwm_output1
INFO  [logger] logger started (mode=all)
INFO  [logger] log root dir created: /fs/microsd/log

NuttShell (NSH)
nsh> 
nsh> INFO  [commander] [cal] calibration started: 2 mag
INFO  [commander] [cal] pending: back front left right up down
INFO  [commander] [cal] hold vehicle still on a pending side
INFO  [commander] [cal] detected rest position, hold still...
INFO  [commander] [cal] down orientation detected
INFO  [commander] [cal] down orientation detected
INFO  [commander] [cal] Rotate vehicle around the detected orientation
INFO  [commander] [cal] Continue rotation for down 7 s
INFO  [commander] [cal] down orientation detected
INFO  [commander] [cal] down orientation detected
INFO  [commander] [cal] Rotate vehicle around the detected orientation
INFO  [commander] [cal] Continue rotation for down 7 s
INFO  [commander] [cal] down side calibration: progress <4>
INFO  [commander] [cal] down side calibration: progress <8>
INFO  [commander] [cal] down side calibration: progress <12>
INFO  [commander] [cal] down side done, rotate to a different side
INFO  [commander] [cal] progress <16>
INFO  [commander] [cal] down side done, rotate to a different side
INFO  [commander] [cal] down side done, rotate to a different side
INFO  [commander] [cal] pending: back front left right up
INFO  [commander] [cal] hold vehicle still on a pending side
INFO  [commander] [cal] detected rest position, hold still...
INFO  [commander] [cal] up orientation detected
INFO  [commander] [cal] up orientation detected
INFO  [commander] [cal] Rotate vehicle around the detected orientation
INFO  [commander] [cal] Continue rotation for up 7 s
INFO  [commander] [cal] up side calibration: progress <16>
INFO  [commander] [cal] up side calibration: progress <20>
INFO  [commander] [cal] up side calibration: progress <24>
INFO  [commander] [cal] up side calibration: progress <28>
INFO  [commander] [cal] up side done, rotate to a different side
INFO  [commander] [cal] progress <33>
INFO  [commander] [cal] up side done, rotate to a different side
INFO  [commander] [cal] up side done, rotate to a different side
INFO  [commander] [cal] pending: back front left right
INFO  [commander] [cal] hold vehicle still on a pending side
INFO  [commander] [cal] detected rest position, hold still...
INFO  [commander] [cal] down side already completed
INFO  [commander] [cal] pending: back front left right
INFO  [commander] [cal] hold vehicle still on a pending side
INFO  [commander] [cal] detected rest position, hold still...
INFO  [commander] [cal] down side already completed
INFO  [commander] [cal] pending: back front left right
INFO  [commander] [cal] hold vehicle still on a pending side
INFO  [commander] [cal] detected rest position, hold still...
INFO  [commander] [cal] down side already completed
INFO  [commander] [cal] pending: back front left right
INFO  [commander] [cal] hold vehicle still on a pending side
INFO  [commander] [cal] detected rest position, hold still...
INFO  [commander] [cal] down side already completed
INFO  [commander] [cal] pending: back front left right
INFO  [commander] [cal] hold vehicle still on a pending side
nsh> ERROR [PreFlightCheck] state 15: |0.00315562| > 0.00173000 + 0.00104747
WARN  [ecl/EKF] baro hgt timeout - reset to baro
ERROR [PreFlightCheck] state 13: |0.00316076| > 0.00173000 + 0.00094887
INFO  [commander] [cal] front side calibration: progress <33>
ERROR [PreFlightCheck] state 13: |0.00314689| > 0.00173000 + 0.00094575
INFO  [commander] [cal] front side calibration: progress <37>
ERROR [PreFlightCheck] state 13: |0.00304873| > 0.00173000 + 0.00094562
ERROR [PreFlightCheck] state 13: |0.00298443| > 0.00173000 + 0.00094546
ERROR [PreFlightCheck] state 13: |0.00280757| > 0.00173000 + 0.00094449
ERROR [PreFlightCheck] state 15: |0.00310471| > 0.00173000 + 0.00100254
INFO  [commander] [cal] front side calibration: progress <41>
INFO  [commander] [cal] front side calibration: progress <45>
INFO  [commander] [cal] front side done, rotate to a different side
INFO  [commander] [cal] progress <50>
INFO  [commander] [cal] front side done, rotate to a different side
INFO  [commander] [cal] front side done, rotate to a different side
INFO  [commander] [cal] pending: back left right
INFO  [commander] [cal] hold vehicle still on a pending side
ERROR [PreFlightCheck] state 13: |0.00312312| > 0.00173000 + 0.00000000
ERROR [PreFlightCheck] state 13: |0.00311632| > 0.00173000 + 0.00000000
ERROR [PreFlightCheck] state 13: |0.00311671| > 0.00173000 + 0.00070686
ERROR [PreFlightCheck] state 13: |0.00313795| > 0.00173000 + 0.00070670
INFO  [commander] [cal] detected rest position, hold still...
INFO  [commander] [cal] down side already completed
INFO  [commander] [cal] pending: back left right
INFO  [commander] [cal] hold vehicle still on a pending side
ERROR [PreFlightCheck] state 13: |0.00314406| > 0.00173000 + 0.00070311
ERROR [PreFlightCheck] state 13: |0.00315083| > 0.00173000 + 0.00069064
INFO  [commander] [cal] detected rest position, hold still...
INFO  [commander] [cal] down side already completed
INFO  [commander] [cal] pending: back left right
INFO  [commander] [cal] hold vehicle still on a pending side
ERROR [PreFlightCheck] state 13: |0.00313979| > 0.00173000 + 0.00067082
INFO  [commander] [cal] right side done, rotate to a different side
INFO  [commander] [cal] progress <83>
INFO  [commander] [cal] right side done, rotate to a different side
INFO  [commander] [cal] right side done, rotate to a different side
ERROR [PreFlightCheck] state 13: |0.00305941| > 0.00173000 + 0.00067082
INFO  [commander] [cal] pending: left
INFO  [commander] [cal] hold vehicle still on a pending side
ERROR [PreFlightCheck] state 13: |0.00296609| > 0.00173000 + 0.00000000
ERROR [PreFlightCheck] state 13: |0.00297235| > 0.00173000 + 0.00000000
ERROR [PreFlightCheck] state 13: |0.00295619| > 0.00173000 + 0.00067085
INFO  [commander] [cal] detected rest position, hold still...
ERROR [PreFlightCheck] state 13: |0.00279319| > 0.00173000 + 0.00067082
INFO  [commander] [cal] left orientation detected
INFO  [commander] [cal] left orientation detected
INFO  [commander] [cal] Rotate vehicle around the detected orientation
INFO  [commander] [cal] Continue rotation for left 7 s
ERROR [PreFlightCheck] state 13: |0.00265631| > 0.00173000 + 0.00067082
INFO  [commander] [cal] left side calibration: progress <83>
WARN  [ecl/EKF] baro hgt timeout - reset to baro
ERROR [PreFlightCheck] state 13: |0.00265622| > 0.00173000 + 0.00067083
INFO  [commander] [cal] left side calibration: progress <87>
ERROR [PreFlightCheck] state 13: |0.00282697| > 0.00173000 + 0.00067082
INFO  [commander] [cal] left side calibration: progress <91>
ERROR [PreFlightCheck] state 13: |0.00264159| > 0.00173000 + 0.00067082
ERROR [PreFlightCheck] state 14: |0.00316500| > 0.00173000 + 0.00067082
INFO  [commander] [cal] left side calibration: progress <95>
ERROR [PreFlightCheck] state 14: |0.00313558| > 0.00173000 + 0.00067082
INFO  [commander] [cal] left side done, rotate to a different side
INFO  [commander] [cal] progress <100>
INFO  [commander] [cal] left side done, rotate to a different side
INFO  [commander] [cal] left side done, rotate to a different side
ERROR [PreFlightCheck] state 14: |0.00313084| > 0.00173000 + 0.00067082
INFO  [commander] [cal] mag #0 off: x:-0.13 y:-0.03 z:-0.01 Ga
INFO  [commander] [cal] mag #0 scale: x:1.01 y:0.96 z:1.02
ERROR [PreFlightCheck] state 13: |0.00175301| > 0.00173000 + 0.00000000
INFO  [commander] [cal] progress <100>
INFO  [commander] [cal] calibration done: mag
ERROR [PreFlightCheck] state 13: |0.00175301| > 0.00173000 + 0.00000000
ERROR [PreFlightCheck] state 14: |0.00311358| > 0.00173000 + 0.00067082
ERROR [PreFlightCheck] state 14: |0.00312592| > 0.00173000 + 0.00067084
ERROR [PreFlightCheck] state 14: |0.00312777| > 0.00173000 + 0.00067082
ERROR [PreFlightCheck] state 13: |0.00267434| > 0.00173000 + 0.00067082
ERROR [PreFlightCheck] state 13: |0.00314079| > 0.00173000 + 0.00067082
WARN  [ecl/EKF] baro hgt timeout - reset to baro
ERROR [PreFlightCheck] state 13: |0.00315811| > 0.00173000 + 0.00067082
ERROR [PreFlightCheck] state 13: |0.00317796| > 0.00173000 + 0.00067082
ERROR [PreFlightCheck] state 13: |0.00316416| > 0.00173000 + 0.00067082
[commander] [cal] calibration started: 2 gyro
INFO  [commander] [cal] progress <4>
INFO  [commander] [cal] progress <9>
INFO  [commander] [cal] progress <14>
INFO  [commander] [cal] progress <19>
INFO  [commander] [cal] progress <24>
INFO  [commander] [cal] progress <28>
INFO  [commander] [cal] progress <33>
INFO  [commander] [cal] progress <38>
INFO  [commander] [cal] progress <43>
INFO  [commander] [cal] progress <48>
INFO  [commander] [cal] progress <52>
INFO  [commander] [cal] progress <57>
INFO  [commander] [cal] progress <62>
INFO  [commander] [cal] progress <67>
INFO  [commander] [cal] progress <72>
INFO  [commander] [cal] progress <76>
INFO  [commander] [cal] progress <81>
INFO  [commander] [cal] progress <86>
INFO  [commander] [cal] progress <91>
INFO  [commander] [cal] progress <96>
WARN  [commander] motion, retrying..
INFO  [commander] [cal] progress <4>
INFO  [commander] [cal] progress <9>
INFO  [commander] [cal] progress <14>
INFO  [commander] [cal] progress <19>
INFO  [commander] [cal] progress <24>
INFO  [commander] [cal] progress <28>
INFO  [commander] [cal] progress <33>
INFO  [commander] [cal] progress <38>
INFO  [commander] [cal] progress <43>
INFO  [commander] [cal] progress <48>
INFO  [commander] [cal] progress <52>
INFO  [commander] [cal] progress <57>
INFO  [commander] [cal] progress <62>
INFO  [commander] [cal] progress <67>
INFO  [commander] [cal] progress <72>
INFO  [commander] [cal] progress <76>
INFO  [commander] [cal] progress <81>
INFO  [commander] [cal] progress <86>
INFO  [commander] [cal] progress <91>
INFO  [commander] [cal] progress <96>
WARN  [commander] motion, retrying..
INFO  [commander] [cal] progress <4>
INFO  [commander] [cal] progress <9>
INFO  [commander] [cal] progress <14>
INFO  [commander] [cal] progress <19>
INFO  [commander] [cal] progress <24>
INFO  [commander] [cal] progress <28>
INFO  [commander] [cal] progress <33>
INFO  [commander] [cal] progress <38>
INFO  [commander] [cal] progress <43>
INFO  [commander] [cal] progress <48>
INFO  [commander] [cal] progress <52>
INFO  [commander] [cal] progress <57>
INFO  [commander] [cal] progress <62>
INFO  [commander] [cal] progress <67>
INFO  [commander] [cal] progress <72>
INFO  [commander] [cal] progress <76>
INFO  [commander] [cal] progress <81>
INFO  [commander] [cal] progress <86>
INFO  [commander] [cal] progress <91>
INFO  [commander] [cal] progress <96>
WARN  [commander] motion, retrying..
INFO  [commander] [cal] progress <4>
INFO  [commander] [cal] progress <9>
INFO  [commander] [cal] progress <14>
INFO  [commander] [cal] progress <19>
INFO  [commander] [cal] progress <24>
INFO  [commander] [cal] progress <28>
INFO  [commander] [cal] progress <33>
INFO  [commander] [cal] progress <38>
INFO  [commander] [cal] progress <43>
INFO  [commander] [cal] progress <48>
INFO  [commander] [cal] progress <52>
INFO  [commander] [cal] progress <57>
INFO  [commander] [cal] progress <62>
INFO  [commander] [cal] progress <67>
INFO  [commander] [cal] progress <72>
INFO  [commander] [cal] progress <76>
INFO  [commander] [cal] progress <81>
INFO  [commander] [cal] progress <86>
WARN  [ecl/EKF] baro hgt timeout - reset to baro
INFO  [commander] [cal] progress <91>
INFO  [commander] [cal] progress <96>
INFO  [commander] [cal] calibration done: gyro
nsh> WARN  [ecl/EKF] baro hgt timeout - reset to baro
INFO  [commander] [cal] calibration started: 2 accel
INFO  [commander] [cal] pending: back front left right up down
INFO  [commander] [cal] hold vehicle still on a pending side
INFO  [commander] [cal] detected rest position, hold still...
INFO  [commander] [cal] down orientation detected
INFO  [commander] [cal] down orientation detected
INFO  [commander] [cal] Hold still, measuring down side
INFO  [commander] [cal] down side result: [  3.3492   3.6232  -6.1397]
INFO  [commander] [cal] progress <17>
INFO  [commander] [cal] down side done, rotate to a different side
INFO  [commander] [cal] down side done, rotate to a different side
INFO  [commander] [cal] pending: back front left right up
INFO  [commander] [cal] hold vehicle still on a pending side
ERROR [PreFlightCheck] state 13: |-0.00315399| > 0.00173000 + 0.00067082
INFO  [commander] [cal] detected rest position, hold still...
ERROR [PreFlightCheck] state 13: |-0.00313378| > 0.00173000 + 0.00067082
INFO  [commander] [cal] detected motion, hold still...
ERROR [PreFlightCheck] state 13: |-0.00312112| > 0.00173000 + 0.00000000
ERROR [PreFlightCheck] state 13: |-0.00312173| > 0.00173000 + 0.00067083
ERROR [PreFlightCheck] state 13: |-0.00314185| > 0.00173000 + 0.00067082
ERROR [PreFlightCheck] state 13: |-0.00313244| > 0.00173000 + 0.00067082
INFO  [commander] [cal] detected rest position, hold still...
ERROR [PreFlightCheck] state 13: |-0.00311766| > 0.00173000 + 0.00067082
ERROR [PreFlightCheck] state 13: |-0.00311401| > 0.00173000 + 0.00067082
INFO  [commander] [cal] up orientation detected
INFO  [commander] [cal] up orientation detected
INFO  [commander] [cal] Hold still, measuring up side
ERROR [PreFlightCheck] state 13: |-0.00312475| > 0.00173000 + 0.00067082
INFO  [commander] [cal] up side result: [  3.8137   4.2622  12.6688]
INFO  [commander] [cal] progress <34>
INFO  [commander] [cal] up side done, rotate to a different side
INFO  [commander] [cal] up side done, rotate to a different side
INFO  [commander] [cal] pending: back front left right
INFO  [commander] [cal] hold vehicle still on a pending side
ERROR [PreFlightCheck] state 13: |-0.00298188| > 0.00173000 + 0.00067082
ERROR [PreFlightCheck] state 13: |-0.00313638| > 0.00173000 + 0.00067082
ERROR [PreFlightCheck] state 13: |-0.00313638| > 0.00173000 + 0.00000000
INFO  [commander] [cal] detected rest position, hold still...
WARN  [ecl/EKF] baro hgt timeout - reset to baro
ERROR [PreFlightCheck] state 13: |-0.00256917| > 0.00173000 + 0.00067082
INFO  [commander] [cal] down side already completed
INFO  [commander] [cal] pending: back front left right
INFO  [commander] [cal] hold vehicle still on a pending side
ERROR [PreFlightCheck] state 14: |0.00314447| > 0.00173000 + 0.00067082
INFO  [commander] [cal] hold vehicle still on a pending side
ERROR [PreFlightCheck] state 14: |0.00314447| > 0.00173000 + 0.00067082
ERROR [PreFlightCheck] state 14: |0.00315848| > 0.00173000 + 0.00067082
ERROR [PreFlightCheck] state 14: |0.00350202| > 0.00173000 + 0.00067082
INFO  [commander] [cal] detected rest position, hold still...
ERROR [PreFlightCheck] state 14: |0.00311158| > 0.00173000 + 0.00067082
INFO  [commander] [cal] down side already completed
INFO  [commander] [cal] pending: back front left right
INFO  [commander] [cal] hold vehicle still on a pending side
ERROR [PreFlightCheck] state 14: |0.00310218| > 0.00173000 + 0.00067082
ERROR [PreFlightCheck] state 14: |0.00310332| > 0.00173000 + 0.00067082
INFO  [commander] [cal] detected rest position, hold still...
ERROR [PreFlightCheck] state 14: |0.00305175| > 0.00173000 + 0.00067082
ERROR [PreFlightCheck] state 14: |0.00274292| > 0.00173000 + 0.00067082
INFO  [commander] [cal] down side already completed
INFO  [commander] [cal] pending: back front left right
INFO  [commander] [cal] hold vehicle still on a pending side
ERROR [PreFlightCheck] state 15: |0.00315654| > 0.00173000 + 0.00067082
ERROR [PreFlightCheck] state 15: |0.00316423| > 0.00173000 + 0.00067082
INFO  [commander] [cal] detected rest position, hold still...
ERROR [PreFlightCheck] state 13: |-0.00255149| > 0.00173000 + 0.00067082
ERROR [PreFlightCheck] state 13: |-0.00313897| > 0.00173000 + 0.00067082
INFO  [commander] [cal] down side already completed
INFO  [commander] [cal] pending: back front left right
INFO  [commander] [cal] hold vehicle still on a pending side
ERROR [PreFlightCheck] state 13: |-0.00317967| > 0.00173000 + 0.00067082
ERROR [PreFlightCheck] state 13: |-0.00316114| > 0.00173000 + 0.00067082
INFO  [commander] [cal] detected rest position, hold still...
ERROR [PreFlightCheck] state 13: |-0.00313994| > 0.00173000 + 0.00067082
INFO  [commander] [cal] detected motion, hold still...
ERROR [PreFlightCheck] state 13: |-0.00313484| > 0.00173000 + 0.00000000
ERROR [PreFlightCheck] state 13: |-0.00313091| > 0.00173000 + 0.00000000
ERROR [PreFlightCheck] state 13: |-0.00311079| > 0.00173000 + 0.00067084
ERROR [PreFlightCheck] state 13: |-0.00313807| > 0.00173000 + 0.00067082
ERROR [PreFlightCheck] state 13: |-0.00313651| > 0.00173000 + 0.00067082
INFO  [commander] [cal] detected rest position, hold still...
ERROR [PreFlightCheck] state 13: |-0.00313159| > 0.00173000 + 0.00067082
ERROR [PreFlightCheck] state 13: |-0.00310030| > 0.00173000 + 0.00067082
INFO  [commander] [cal] front orientation detected
INFO  [commander] [cal] front orientation detected
INFO  [commander] [cal] Hold still, measuring front side
ERROR [PreFlightCheck] state 13: |-0.00278395| > 0.00173000 + 0.00067082
INFO  [commander] [cal] front side result: [ -6.3692   3.8747   3.4852]
INFO  [commander] [cal] progress <51>
INFO  [commander] [cal] front side done, rotate to a different side
INFO  [commander] [cal] front side done, rotate to a different side
INFO  [commander] [cal] pending: back left right
INFO  [commander] [cal] hold vehicle still on a pending side
ERROR [PreFlightCheck] state 14: |0.00306755| > 0.00173000 + 0.00067082
ERROR [PreFlightCheck] state 14: |0.00303840| > 0.00173000 + 0.00067082
ERROR [PreFlightCheck] state 14: |0.00309299| > 0.00173000 + 0.00000000
ERROR [PreFlightCheck] state 14: |0.00309823| > 0.00173000 + 0.00067085
ERROR [PreFlightCheck] state 14: |0.00314014| > 0.00173000 + 0.00067082
ERROR [PreFlightCheck] state 14: |0.00313463| > 0.00173000 + 0.00067082
INFO  [commander] [cal] detected rest position, hold still...
ERROR [PreFlightCheck] state 14: |0.00314528| > 0.00173000 + 0.00067082
ERROR [PreFlightCheck] state 13: |0.00316540| > 0.00173000 + 0.00067082
INFO  [commander] [cal] back orientation detected
INFO  [commander] [cal] back orientation detected
INFO  [commander] [cal] Hold still, measuring back side
ERROR [PreFlightCheck] state 13: |0.00316758| > 0.00173000 + 0.00067082
INFO  [commander] [cal] back side result: [ 13.3974   3.5423   3.3356]
INFO  [commander] [cal] progress <68>
INFO  [commander] [cal] back side done, rotate to a different side
INFO  [commander] [cal] back side done, rotate to a different side
INFO  [commander] [cal] pending: left right
INFO  [commander] [cal] hold vehicle still on a pending side
ERROR [PreFlightCheck] state 13: |0.00313111| > 0.00173000 + 0.00067082
ERROR [PreFlightCheck] state 13: |0.00313126| > 0.00173000 + 0.00067082
ERROR [PreFlightCheck] state 13: |0.00312711| > 0.00173000 + 0.00000000
ERROR [PreFlightCheck] state 13: |0.00311000| > 0.00173000 + 0.00067082
ERROR [PreFlightCheck] state 13: |0.00309408| > 0.00173000 + 0.00067082
nsh> INFO  [commander] [cal] calibration started: 2 level
INFO  [commander] [cal] progress <0>
INFO  [commander] [cal] progress <20>
INFO  [commander] [cal] progress <40>
INFO  [commander] [cal] progress <60>
INFO  [commander] [cal] progress <80>
INFO  [commander] [cal] progress <100>
INFO  [commander] [cal] progress <100>
INFO  [commander] [cal] calibration done: level

sensors status:

nsh> sensors status
INFO  [sensors] gyro status:
INFO  [ecl/validation] validator: best: 0, prev best: 0, failsafe: NO (0 events)
INFO  [ecl/validation] sensor #0, prio: 75, state: OK
INFO  [ecl/validation]  val:   0.0010, lp:   0.0006 mean dev:   0.0001 RMS:   0.5176 conf:   1.0000
INFO  [ecl/validation]  val:  -0.0032, lp:   0.0000 mean dev:  -0.0001 RMS:   0.3976 conf:   1.0000
INFO  [ecl/validation]  val:  -0.0106, lp:  -0.0005 mean dev:   0.0001 RMS:   0.4126 conf:   1.0000
INFO  [sensors] accel status:
INFO  [ecl/validation] validator: best: 0, prev best: 0, failsafe: NO (0 events)
INFO  [ecl/validation] sensor #0, prio: 75, state: OK
INFO  [ecl/validation]  val:   0.4276, lp:   0.4823 mean dev:  -0.0020 RMS:   0.9222 conf:   1.0000
INFO  [ecl/validation]  val:   0.1706, lp:   0.0710 mean dev:  -0.0024 RMS:   1.0379 conf:   1.0000
INFO  [ecl/validation]  val:  -9.5963, lp:  -9.8102 mean dev:  -0.0024 RMS:   1.3680 conf:   1.0000
INFO  [sensors] mag status:
INFO  [ecl/validation] validator: best: 0, prev best: 0, failsafe: NO (0 events)
INFO  [ecl/validation] sensor #0, prio: 50, state: OK
INFO  [ecl/validation]  val:   0.0116, lp:   0.0109 mean dev:   0.0006 RMS:   0.0898 conf:   1.0000
INFO  [ecl/validation]  val:  -0.3680, lp:  -0.3688 mean dev:  -0.0009 RMS:   0.0934 conf:   1.0000
INFO  [ecl/validation]  val:   0.3354, lp:   0.3355 mean dev:   0.0000 RMS:   0.0929 conf:   1.0000

INFO  [sensors] Airspeed status:
INFO  [ecl/validation]  no data

INFO  [vehicle_acceleration] selected sensor: 1114122 (0)
INFO  [vehicle_acceleration] bias: [-0.157 -0.095 -0.117]
INFO  [vehicle_acceleration] sample rate: 777.384 Hz

INFO  [vehicle_angular_velocity] selected sensor: 2228234 (0)
INFO  [vehicle_angular_velocity] bias: [0.000 0.001 -0.004]
INFO  [vehicle_angular_velocity] sample rate: 743.285 Hz

INFO  [vehicle_air_data] selected barometer: 4063242 (0)
vehicle_air_data: cycle: 52313 events, 2783832us elapsed, 53.21us avg, min 18us max 326us 43.336us rms
INFO  [ecl/validation] validator: best: 0, prev best: 0, failsafe: NO (0 events)
INFO  [ecl/validation] sensor #0, prio: 75, state: OK
INFO  [ecl/validation]  val: 204931.0000, lp: 204933.3906 mean dev:  -0.2243 RMS:   5.3842 conf:   1.0000
INFO  [ecl/validation]  val:  33.5800, lp:  33.5872 mean dev:   0.0040 RMS:   0.0101 conf:   1.0000
INFO  [ecl/validation]  val:   0.0000, lp:   0.0000 mean dev:   0.0000 RMS:   0.0000 conf:   1.0000

listener sensor_accel:

TOPIC: sensor_accel
 sensor_accel_s
        timestamp: 732537313  (0.003769 seconds ago)
        timestamp_sample: 732537295
        device_id: 1114122 (Type: 0x11, SPI:1 (0x00)) 
        x: -0.4727
        y: -0.0944
        z: -9.7176
        temperature: 34.2500

listener sensor_gyro:

TOPIC: sensor_gyro
 sensor_gyro_s
        timestamp: 759667571  (0.004555 seconds ago)
        timestamp_sample: 759667554
        device_id: 2228234 (Type: 0x22, SPI:1 (0x00)) 
        x: -0.0160
        y: 0.0007
        z: 0.0009
        temperature: 26.0000

work_queue status:

Work Queue: 6 threads                      RATE        INTERVAL
|__ 1) wq:lp_default   
|   |__ 1) send_event                   30.0 Hz      33330.2 us (33333 us)
|   |__ 2) load_mon                      1.0 Hz     998553.1 us (1000000 us)
|   \__ 3) mc_hover_thrust_estimator      0.0 Hz          0.0 us
|__ 2) wq:hp_default   
|   |__ 1) tone_alarm                   10.2 Hz      98469.8 us
|   |__ 2) rc_update                     0.0 Hz          0.0 us
|   |__ 3) adc                         100.0 Hz       9999.8 us (10000 us)
|   |__ 4) battery_status              100.0 Hz       9999.8 us (10000 us)
|   \__ 5) fmu                           0.0 Hz  195050592.0 us
|__ 3) wq:I2C2         
|   \__ 1) rgbled                       38.1 Hz      26225.4 us
|__ 4) wq:SPI1         
|   |__ 1) ms5611                       99.5 Hz      10054.1 us
|   |__ 2) l3gd20                     1398.6 Hz        715.0 us (715 us)
|   \__ 3) lsm303d                     952.4 Hz       1050.0 us (1050 us)
|__ 5) wq:att_pos_ctrl 
|   |__ 1) sensors                     218.9 Hz       4567.5 us
|   |__ 2) vehicle_acceleration        777.8 Hz       1285.6 us
|   |__ 3) vehicle_air_data             74.6 Hz      13406.5 us
|   |__ 4) vehicle_imu                 428.7 Hz       2332.5 us
|   |__ 5) ekf2                        218.9 Hz       4567.5 us
|   |__ 6) mc_att_control              185.0 Hz       5406.7 us
|   |__ 7) mc_pos_control               33.2 Hz      30124.2 us
|   \__ 8) land_detector                50.0 Hz      19999.3 us (20000 us)
\__ 6) wq:rate_ctrl    
    |__ 1) vehicle_angular_velocity    742.7 Hz       1346.5 us
    \__ 2) mc_rate_control             323.6 Hz       3089.8 us
rolandash commented 4 years ago

@dagar I need to add an update from my testings, might be helpful: Under v1.11.0 mpu6500 seems also have the same behavior as lsm303d & l3gd20. mpu6k works good. Under v1.10.0 I use my previous PR of 9250 driver for mpu6500 and it works just fine. So seems like a reasonable guess that there's something not right with the new driver.

dagar commented 4 years ago

@rolandash I believe the problem here is the estimator not initializing with the data from the lsm303d + l3gd20 combo. Does it work if you do a clean accel + gyro calibration, reboot, then try level horizon?

rolandash commented 4 years ago

@dagar My testings confirms with your judgement. If I do level horizon after sensor cal + reboot, then it works fine.

Expecting the fix from you soon.

dagar commented 4 years ago

@rolandash could you please give current master another try? It's about to be tagged as v1.11.0-beta2. If it's still a problem a log from boot (SDLOG_PROFILE) would help.

rolandash commented 4 years ago

@dagar Just saw your message. Will test it and feedback to you in one day or two.

rolandash commented 4 years ago

@dagar unfortunately the problem seems still around In my testings. I will upload logs later soon.

rolandash commented 4 years ago

@dagar

Please check the SDLOG attached.

log001.zip

stale[bot] commented 4 years ago

This issue has been automatically marked as stale because it has not had recent activity. Thank you for your contributions.

rolandash commented 4 years ago

Hi @dagar

Is this issue considered been addressed in latest 1.11 ?

dagar commented 4 years ago

Hi @dagar

Is this issue considered been addressed in latest 1.11 ?

Well yes and no.

It appears to be an issue with ekf2 where if the IMU has a sufficiently large offset it won't initialize. If you calibrate first, the estimator will initialize (and start publishing vehicle_attitude) and the level horizon calibration will then work.

We could open a PX4/ecl bug and investigate it there, but I'm wondering if we simply need a better error message to suggest calibrating before trying again.

ryanjAA commented 4 years ago

I loaded up 1.11 RC3 zeroed all accel, gyro and mag calibrations and calibrated all in one go including leveling the horizon and it worked. Sensors onboard were LSM303D for compass (1114122), MPU600 for gyro (2162698). Gyro 2 as L3GD20 (2228234). Offsets would have been zero since I did it all from zero but it does work. Soft reboot and leveled again and leveled fine. No external sensors. All internal.

stale[bot] commented 3 years ago

This issue has been automatically marked as stale because it has not had recent activity. Thank you for your contributions.

ryanjAA commented 3 years ago

So this is interestingly back. On release 1.11.2 if i follow the above and calibrate gyro and accel first, i can calibrate the horizon but once i reboot i cant calibrate it. Looking at the offsets of the accels, one of them has offsets of around 3 (MRO Pixhawk and i tried 3 different ones).

Presumably all hardware will have an offset different between two different sensors so this not being able to cope with that is probably the change needed.

At the same time it isn't really feasible to have to calibrate gyro and accel every time you want to level the horizon (mag doesn't seem to matter here).

The shortcut I've been using is if you hit calibrate on the horizon and let it sit for a few seconds, it will get to the end, just not complete. Then exiting out the calibration pane (not hitting cancel) and checking the Sens board offset has the new offsets in there. That seems to get the level horizon done as a hack.

dagar commented 3 years ago

It should be better in master now (what will be v1.12), I relaxed the acceleration threshold slightly for ecl/EKF initialization not too long ago. https://github.com/PX4/PX4-ECL/pull/931/files#diff-a002684e5addc90ff5d584ea8d7c0061104b10c15112b3d28f8bf34f0742deb2R242-R243

ryanjAA commented 3 years ago

OK Cool - Will the hack im currently doing be sufficient though for leveling in the mean time?

dagar commented 3 years ago

OK Cool - Will the hack im currently doing be sufficient though for leveling in the mean time?

I would say yes only if you're not having to redo it more than once per vehicle.

What's happening is the lsm303d + l3gd20 combo is being given the same initial priority as the mpu6000, but since it has a much faster reset it ends up publishing data first and becomes the selected sensor. I would either lower the priority of the lsm303d + l3gd20 (CAL_ACCx_PRIO/CAL_GYROx_PRIO) or in startup add a delay between starting the mpu6000 and lsm303d + l3gd20 so that you get the mpu6000 first (which has much smaller offsets).

All of this becomes irrelevant with Multi-EKF.

ryanjAA commented 3 years ago

I'll try lowering the priority right now.

It happens to every vehicle on 1.11.2 after every reboot.

Excited about multi-ekf, just want to get this useable in the mean time.

ryanjAA commented 3 years ago

Ah - 1.11.2 doesn't have sensor priority yet.

Ok so how to delay them and by how much?

LorenzMeier commented 3 years ago

@dagar This sounds though like the condition for sensor usage in EKF2 should be reset once the system goes out of calibration mode.

dagar commented 3 years ago

@dagar This sounds though like the condition for sensor usage in EKF2 should be reset once the system goes out of calibration mode.

It already works that way, it's just really unintuitive if you fall into that hole because there's nothing that directly connects the underlying problem and what to do about it.

To sidestep the issue I bumped the threshold a few weeks ago. https://github.com/PX4/PX4-ECL/blob/18f334f4dbca61e3bcd1743c22cf820b5a8ff7ef/EKF/ekf.cpp#L239-L240

The vast majority of sensors we support are nowhere need bad enough to have this problem.

ryanjAA commented 3 years ago

If ecl/ekf is refusing to initialize is this a problem for flight or only for sensor calibration?

Meaning if the horizon doesn’t need to be leveled again as it was done previously are there concerns ekf still won’t initialize?

dagar commented 3 years ago

If ecl/ekf is refusing to initialize is this a problem for flight or only for sensor calibration?

It's only a problem at initialization/startup, and only for particularly bad sensors old sensors.

ryanjAA commented 3 years ago

Thanks. Was worried for a moment to fly with them.

ryanjAA commented 3 years ago

As @dagar pointed out here https://px4.slack.com/archives/C0V533X4N/p1609285210239700?thread_ts=1609283837.239400&cid=C0V533X4N adding a sleep 1 to the line fixes this but just remember that when you startup you need to wait ~3 seconds before leveling the horizon or you will get "attitude estimator not running - check system boot”. Attached is the binary compiled (and compressed to GitHub accepts the file type) so you can just flash it. No other changes made and built from 1.11.2 release px4_fmu-v3_default.px4.zip

ryanjAA commented 3 years ago

Could this somehow be related to how I'm connected?

When I try and level the horizon via USB it works fine but when wireless it does not.

@dagar

ryanjAA commented 3 years ago

So if I set MAV_0_rate to 1200 it doesn't even finish (when connected wirelessly). When I set it to 0 (aka half of max), the vehicles beeps that it is complete but QGC hangs at the very end of the cal.