search

ArduPilot / ardupilot

ArduPlane, ArduCopter, ArduRover, ArduSub source
http://ardupilot.org/
GNU General Public License v3.0
10.97k stars 17.5k forks source link

Internal Error Reporting Page #15916

Open Hwurzburg opened 3 years ago

Hwurzburg commented 3 years ago

This page is for anyone to report an Internal Error ("Internal Error:" in GCS or DataFlash messages). This allows the Dev team to be alerted, and to investigate as soon as possible.

Please enter the firmware used, vehicle type, and the Internal Error message. If you have a log, please retain it for 30 days in case a Developer would like to examine it.

Note: follow up timing on a new issue will depend on many factors, so some may not be addressed immediately.

The list of internal error numbers and meaning can be found here.

jini93 commented 3 years ago

hi I'm Korean. Let me ask you a question. Please understand if I say something weird My message is like a picture.) 화면 캡처 2020-12-04 161250 화면 캡처 2020-12-04 161250

ndotkarkut commented 3 years ago

I receive Internal Error 0x1000. Pixhawk1 4.0.5 Stable Firmware Build. HexaCopter.

peterbarker commented 3 years ago

hi I'm Korean. Let me ask you a question. Please understand if I say something weird My message is like a picture.) 화면 캡처 2020-12-04 161250 화면 캡처 2020-12-04 161250

What version of the firmware was this with? A git hash is prefered.

peterbarker commented 3 years ago

I receive Internal Error 0x1000. Pixhawk1 4.0.5 Stable Firmware Build. HexaCopter.

When did this happen? Close to boot time? After doing something with the device (e.g. flashing a bootloader)?

Just after upgrading to 4.0.5?

alvinc103 commented 3 years ago

Internal Error message: PREARM: INTERNAL ERRORS (0x4000) Firmware used: Arduplane V4.0.7 Vehicle type: Wing

Internal error

Prearm error log: https://www.dropbox.com/s/qnyhvh19c59f4rf/2020-12-09%2009-16-03.bin?dl=0

After I cycled the power I was able to arm, but the plane crashed during takeoff. It banked to the left and crashed. Not sure if it is related to the error but it would be great to find out what caused the crash.

Crash log: https://www.dropbox.com/s/i862unbg7bw45bi/2020-12-09%2009-21-51.bin?dl=0

Thanks

giacomo892 commented 3 years ago

Matek F405-STD I was downloading logs from SDCARD via MAVLink running master as 4 jan image

Hwurzburg commented 3 years ago

Attempting to re-arm after landing a TVBS using MatekH743 arming was prevented by an internal error.....it has IMU0 disabled since the ICM20602 drifts over time...only IMU1 is used....

PreArm:INTERNAL ERRORS 0X1000000 L:160 IMU_RESET

bnsgeyer commented 3 years ago

@peterbarker @rmackay9 @tridge Reporting the internal error that resulted in a hard fault and watchdog reset. This has occurred twice for this user with a usable watchdog reset code. The first time it occurred was reported in this discuss post and the second time reported in the same thread but here is the post. In speaking with Peter Barker about this, he seems to think there is an issue with IRQ handling. receiver is FrSky R-XSR, connected via S-BUS to the PIXHAWK 4 mini.

User reported that he had flown master on this particular setup with no issues prior to loading 4.0.6.

User setup details: 500 size Heli: MSG, 648550, ArduCopter V4.0.6-ATNH-rc1 (780c4966) MSG, 648566, ChibiOS: d4fce84e MSG, 648590, PH4-mini 003F0027 31385118 35343633 This code is a version of heli autotune built on the stable version of 4.0.6. branch is here

Log files after the watchdog reset are here. ELF and object dump for latest log is in the same link. The previous log used the same branch built on an older compiler version and I didn't keep the elf or object dump.

Here is the decoded WDOG msg pbarker@bluebottle:~/rc/ardupilot(pr/decode-watch)$ ./Tools/scripts/decode_watchdog.py "WDOG, 2641424, -3, 0, 0, 0, 0, 0, 0, 122, 3, 0, 181, 4196355, 135203219, SPI1" T Scheduler Task: -3: Waiting for sample IE Internal Error Mask: 0: ????? IEC Internal Error Count: 0: 0 IEL Internal Error Line: 0: 0 MM MAVLink Message: 0: [None] MC MAVLink Command: 0: [None] SL Semaphore Line: 0: Not waiting on semaphore FL Fault Line: 122: ????? FT Fault Type: 3: HardFault FA Fault Address: 0x0: ????? FTP Fault Thread Priority: 181: ????? FICSR Fault ICS Register: 0x4196355: [Below] VECTACTIVE: 3 (Hard fault) RESERVED1: 0 RETOBASE: 1 (no (or no more) active exceptions) VECTPENDING: 0 (Thread mode) RESERVED2: 0 ISRPENDING: 1 (Interrupt pending) RESERVED3: 0 PENDSTCLR: 0 (WO clears SysTick exception) PENDSTSET: 0 (SysTick not pending) PENDSVCLR: 0 (WO clears pendsv exception) PENDSVSET: 0 (SysTick not pending) RESERVED4: 0 NMIPENDSET: 0 (NMI not pending) FLR Fault Long Return Address: 0x80f0993: ????? TN Thread name: SPI1: ?????

and here is the portion of code the Fault address points to 80f0982: e7e2 b.n 80f094a <_ZN7ChibiOS9SPIDevice8transferEPKhmPhm+0x66> 80f0984: 44d8 add r8, fp 80f0986: 2100 movs r1, #0 80f0988: 462a mov r2, r5 80f098a: 607b str r3, [r7, #4] 80f098c: 4640 mov r0, r8 80f098e: f022 f959 bl 8112c44 80f0992: 4630 mov r0, r6 80f0994: 4659 mov r1, fp 80f0996: 465a mov r2, fp 80f0998: 687b ldr r3, [r7, #4] 80f099a: f7ff fee5 bl 80f0768 <_ZN7ChibiOS9SPIDevice11do_transferEPKhPhm> 80f099e: 2800 cmp r0, #0 80f09a0: d0d9 beq.n 80f0956 <_ZN7ChibiOS9SPIDevice8transferEPKhmPhm+0x72>

For the second watchdog reset, here is the WDOG message decoded ./Tools/scripts/decode_watchdog.py "WDOG {TimeUS : 183209104, Tsk : -3, IE : 0, IEC : 0, IEL : 0, MvMsg : 0, MvCmd : 0, SmLn : 0, FL : 122, FT : 3, FA : 0, FP : 177, ICSR : 4450307, LR : 135314015, TN : rcin}" T Scheduler Task: -3: Waiting for sample IE Internal Error Mask: 0: ????? IEC Internal Error Count: 0: 0 IEL Internal Error Line: 0: 0 MM MAVLink Message: 0: [None] MC MAVLink Command: 0: [None] SL Semaphore Line: 0: Not waiting on semaphore FL Fault Line: 122: ????? FT Fault Type: 3: HardFault FA Fault Address: 0x0: ????? FTP Fault Thread Priority: 177: ????? FICSR Fault ICS Register: 0x4450307: [Below] VECTACTIVE: 3 (Hard fault) RESERVED1: 0 RETOBASE: 1 (no (or no more) active exceptions) VECTPENDING: 62 (IRQ46) RESERVED2: 0 ISRPENDING: 1 (Interrupt pending) RESERVED3: 0 PENDSTCLR: 0 (WO clears SysTick exception) PENDSTSET: 0 (SysTick not pending) PENDSVCLR: 0 (WO clears pendsv exception) PENDSVSET: 0 (SysTick not pending) RESERVED4: 0 NMIPENDSET: 0 (NMI not pending) FLR Fault Long Return Address: 0x135314015: ????? TN Thread name: rcin: ?????

The HEX value is 0x810ba5f and the section of code is below

if (port_irq_enabled(sts)) { 810ba3e: 4605 mov r5, r0 810ba40: b978 cbnz r0, 810ba62 <chSysRestoreStatusX+0x26> __ASM volatile ("MRS %0, ipsr" : "=r" (result) ); 810ba42: f3ef 8405 mrs r4, IPSR 810ba46: f3c4 0408 ubfx r4, r4, #0, #9 if (port_is_isr_context()) { 810ba4a: b124 cbz r4, 810ba56 <chSysRestoreStatusX+0x1a> _stats_stop_measure_crit_isr(); 810ba4c: f7ff ff2e bl 810b8ac <_stats_stop_measure_crit_isr> __ASM volatile ("MSR basepri, %0" : : "r" (basePri) : "memory"); 810ba50: f385 8811 msr BASEPRI, r5 810ba54: bd38 pop {r3, r4, r5, pc} chSysUnlockFromISR(); } else { chSchRescheduleS(); 810ba56: f7ff fe69 bl 810b72c _stats_stop_measure_crit_thd(); 810ba5a: f7ff ff1b bl 810b894 <_stats_stop_measure_crit_thd> 810ba5e: f384 8811 msr BASEPRI, r4 810ba62: bd38 pop {r3, r4, r5, pc}

zhaodelong commented 3 years ago

I reported a internalerror 0x800 incident at https://discuss.ardupilot.org/t/arduplane-4-0-9-froze-in-the-middle-of-flight/69140/2

firmware used: arduplane 4.0.9 stable, vehicle type: plane the WDG message: decoded message in the forum link

Could you investigate this? logs and videos in the forum

axelisart commented 3 years ago

Hi, I (seems randomly but I'm not sure) often got Internal Error 0x1000000 (IMU_RESET). I use a Matek H743 FC in a Tricopter Frame. This started to happen in Arducopter 4.1. Never got this before in 4.0.6 (I migrated to 4.1 based on Matek Support advice as I got lot of ACCELS Inconsistent and Gyro Health problems...). Connected peripherals:

Error Screenshot :

Capture d’écran 2021-03-24 à 13 22 03

Frame Picture for reference (7 inch props) : frame

Target0815 commented 3 years ago

The combination of ArduPlane 4.1 DEV on a H743-Wing is definitely not ok ...

I am currently building a plane with this combination. There are the following problems:

The setup goes corrupt after a reboot. There are currently 98 backup files (STRG0.bak - STR98.bak) in the APM/STRG_BAK directory of the SD card. Here probably a copy was inserted, in order to find the problem.

When trying to arm via MP (Data / Actions / Arm/Disarm) there is immediately an error: "PreArm: Internal errors 0x800000I:57 stack_ovrflw". You can force arming here, but not disarming anymore ...

Screenshot 2021-03-29 092647

I have attached the last 5 backup files.

Matek H743-Wing.zip

rain-er commented 3 years ago

Rover 4.1.0-dev "latest" on Matek H743 When arming/disarming:

giacomo892 commented 3 years ago

FIX IS IN MASTER

Copter master 1fe1e532607de07a38ff9eed44dbb3ae9d333a64 Matek F405-STD LOG_DISARM=1 EKF_REPLAY=1

STABILIZE> log list STABILIZE> Requesting log list Log 1 numLogs 2 lastLog 2 size 1828703 Tue Jan 1 01:00:00 1980 Log 2 numLogs 2 lastLog 2 size 1568768

STABILIZE> log erase STABILIZE> APM: PreArm: Throttle below Failsafe APM: PreArm: Battery 1 low voltage failsafe APM: PreArm: Internal errors 0x800000 l:59 stack_ovrflw

looks like storage thread stack overflow when deleting the logs on sdcard:

ThreadsV2 ISR PRI=255 sp=0x20000000 STACK=1296/1536 ArduCopter PRI=182 sp=0x20000600 STACK=5120/7168 idle PRI= 1 sp=0x200115E8 STACK=144/352 UART_RX PRI= 60 sp=0x2001DBE8 STACK=768/1104 OTG1 PRI= 60 sp=0x2001D308 STACK=264/656 monitor PRI=183 sp=0x2000CF48 STACK=472/848 timer PRI=181 sp=0x2000E168 STACK=1488/1872 rcout PRI=181 sp=0x2000D858 STACK=488/848 rcin PRI=177 sp=0x2000D2D0 STACK=1072/1360 io PRI= 58 sp=0x2000C5C0 STACK=1416/2384 storage PRI= 59 sp=0x2000DBE0 STACK=848/1360 UART1 PRI= 60 sp=0x2001C9B0 STACK=376/656 UART3 PRI= 60 sp=0x2001C150 STACK=256/656 UART4 PRI= 60 sp=0x2001B308 STACK=256/656 I2C0 PRI=176 sp=0x2001A728 STACK=896/1360 FrSky PRI=178 sp=0x20018E28 STACK=1040/1360 UART5 PRI= 60 sp=0x20018700 STACK=304/656 OSD PRI= 59 sp=0x20014210 STACK=664/1360 log_io PRI= 59 sp=0x20014910 STACK=48/1360 SPI1 PRI=181 sp=0x20013530 STACK=656/1360 FTP PRI= 58 sp=0x100088C0 STACK=1704/2896

juvinski commented 3 years ago

Hi guys,

I updated a qplane with ArduPlane V4.1.0dev (31c889f5) - I was with 4.0.8 and after that - I'm running on a bbbmini - beaglebone green + bbbmini cape - I got my linux with more than 16.8 of load average and with 100% of cpu and memory usage. I'm having this error message: 12/04/2021 15:07:38 : PreArm: Internal errors 0x1000000 l:168 imu_reset 12/04/2021 15:07:38 : PreArm: Bad GPS Position 12/04/2021 15:07:38 : PreArm: Compass not healthy 12/04/2021 15:07:28 : PreArm: Internal errors 0x1000000 l:168 imu_reset

I don't know if the load is caused by this imu_reset error or the error is caused by the high consuption. with the 4.0.8 was working fine and the load was around 2%. I'm having this message to: MPU: temp reset IMU[0] 7668 9

andyp1per commented 3 years ago

Copter master 1fe1e53 Matek F405-STD LOG_DISARM=1 EKF_REPLAY=1

STABILIZE> log list STABILIZE> Requesting log list Log 1 numLogs 2 lastLog 2 size 1828703 Tue Jan 1 01:00:00 1980 Log 2 numLogs 2 lastLog 2 size 1568768

STABILIZE> log erase STABILIZE> APM: PreArm: Throttle below Failsafe APM: PreArm: Battery 1 low voltage failsafe APM: PreArm: Internal errors 0x800000 l:59 stack_ovrflw

looks like storage thread stack overflow when deleting the logs on sdcard:

ThreadsV2 ISR PRI=255 sp=0x20000000 STACK=1296/1536 ArduCopter PRI=182 sp=0x20000600 STACK=5120/7168 idle PRI= 1 sp=0x200115E8 STACK=144/352 UART_RX PRI= 60 sp=0x2001DBE8 STACK=768/1104 OTG1 PRI= 60 sp=0x2001D308 STACK=264/656 monitor PRI=183 sp=0x2000CF48 STACK=472/848 timer PRI=181 sp=0x2000E168 STACK=1488/1872 rcout PRI=181 sp=0x2000D858 STACK=488/848 rcin PRI=177 sp=0x2000D2D0 STACK=1072/1360 io PRI= 58 sp=0x2000C5C0 STACK=1416/2384 storage PRI= 59 sp=0x2000DBE0 STACK=848/1360 UART1 PRI= 60 sp=0x2001C9B0 STACK=376/656 UART3 PRI= 60 sp=0x2001C150 STACK=256/656 UART4 PRI= 60 sp=0x2001B308 STACK=256/656 I2C0 PRI=176 sp=0x2001A728 STACK=896/1360 FrSky PRI=178 sp=0x20018E28 STACK=1040/1360 UART5 PRI= 60 sp=0x20018700 STACK=304/656 OSD PRI= 59 sp=0x20014210 STACK=664/1360 log_io PRI= 59 sp=0x20014910 STACK=48/1360 SPI1 PRI=181 sp=0x20013530 STACK=656/1360 FTP PRI= 58 sp=0x100088C0 STACK=1704/2896

Its not the storage thread its the log_io thread

tridge commented 3 years ago

Its not the storage thread its the log_io thread

thanks Andy, I've opened a PR to fix it @giacomo892 please test https://github.com/ArduPilot/ardupilot/pull/17178

peterbarker commented 3 years ago

@tridge did you have a callpath in mind when you thought increasing the stack size might be the solution to this problem?

peterbarker commented 3 years ago

Rover 4.1.0-dev "latest" on Matek H743 When arming/disarming:

* Prearm: Param storage failed

* followed by: InternalError: 0x4000020

We believe this is failure to honour the sd-card-slowdown parameter.

A patch is being tested.

rain-er commented 3 years ago

Rover 4.1.0-dev "latest" on Matek H743 When arming/disarming:

* Prearm: Param storage failed

* followed by: InternalError: 0x4000020

We believe this is failure to honour the sd-card-slowdown parameter.

A patch is being tested.

This was fixed with: https://github.com/ArduPilot/ardupilot/pull/17117 https://discuss.ardupilot.org/t/h743-wing-rover-not-saving-changes-after-power-down/69442

SergeyBokhantsev commented 3 years ago

PreArm: Internal errors 0x800000 l:57 stack_ovrflw if set VTX_ENABLE 1

5/15/2021 10:49:30 PM : PreArm: Internal errors 0x800000 l:57 stack_ovrflw 5/15/2021 10:49:30 PM : PreArm: Throttle below Failsafe 5/15/2021 10:49:07 PM : u-blox 1 HW: 00080000 SW: ROM CORE 3.01 (107888) 5/15/2021 10:49:05 PM : GPS: u-blox 1 saving config 5/15/2021 10:49:01 PM : EKF3 IMU0 origin set 5/15/2021 10:49:00 PM : PreArm: Internal errors 0x800000 l:57 stack_ovrflw 5/15/2021 10:49:00 PM : PreArm: Throttle below Failsafe 5/15/2021 10:48:51 PM : GPS 1: detected as u-blox at 230400 baud 5/15/2021 10:48:49 PM : Frame: QUAD/X 5/15/2021 10:48:49 PM : IMU0: fast sampling enabled 8.0kHz/2.0kHz 5/15/2021 10:48:49 PM : RCOut: DS150:1-4 NeoP:5 5/15/2021 10:48:49 PM : BeastF7 00480036 3039510B 34323433 5/15/2021 10:48:49 PM : ChibiOS: c82ce90e 5/15/2021 10:48:49 PM : ArduCopter V4.1.0-beta1 (a47823a5)

VTX settings (TBS Unify Pro 5.8G HV - Race V2): image @andyp1per

wazoo22 commented 3 years ago

using an early pixhawk 1, delta wing, latest pixhawk 1M firmware, I also get: Pre|Arm: internal error 0x3000 and can't arm.

annoyinganime commented 3 years ago

Matek f405wse (MATEK F405 WING firmware), arducopter v4.1.0-dev. After arming and disarming quadcopter will not arm again, mission planner shows 0x100000 l:868 (flow_of_control) error. After rebooting quad will fly again one time, and fall back to that error again after disarming.

Update: after reading some posts here, I tried removing SD card and disabled logging completely: and it worked! I am now able to disarm and re-arm my quad, and get no error messages. But what to do with logging?

Sekilsgs2 commented 3 years ago

Mamba f405 mk2 ARDU 4.1 beta 3

02.06.2021 11:34:55 : PreArm: Internal errors 0x800000 l:183 stack_ovrfl
02.06.2021 11:34:55 : PreArm: Radio failsafe on
02.06.2021 11:34:25 : w
02.06.2021 11:34:25 : PreArm: Internal errors 0x800000 l:183 stack_ovrfl
02.06.2021 11:34:25 : PreArm: Radio failsafe on
02.06.2021 11:33:56 : w
02.06.2021 11:33:56 : PreArm: Internal errors 0x800000 l:183 stack_ovrfl
02.06.2021 11:33:56 : PreArm: Radio failsafe on
02.06.2021 11:33:26 : w
02.06.2021 11:33:26 : PreArm: Internal errors 0x800000 l:183 stack_ovrfl
02.06.2021 11:33:26 : PreArm: Radio failsafe on
02.06.2021 11:33:17 : EKF3 IMU0 initial yaw alignment complete
02.06.2021 11:33:17 : EKF3 IMU0 tilt alignment complete
02.06.2021 11:33:16 : EKF3 IMU0 initialised
02.06.2021 11:33:16 : EKF3 IMU0 forced reset
02.06.2021 11:33:11 : EKF3 IMU0 initial yaw alignment complete
02.06.2021 11:33:11 : EKF3 IMU0 tilt alignment complete
02.06.2021 11:32:53 : EKF3 IMU0 initialised
02.06.2021 11:32:53 : EKF3 IMU0 forced reset
02.06.2021 11:32:48 : EKF3 IMU0 initial yaw alignment complete
02.06.2021 11:32:48 : EKF3 IMU0 tilt alignment complete
02.06.2021 11:32:45 : EKF3 IMU0 initialised
02.06.2021 11:32:45 : EKF3 IMU0 forced reset
02.06.2021 11:32:39 : EKF3 IMU0 initial yaw alignment complete
02.06.2021 11:32:39 : EKF3 IMU0 tilt alignment complete
02.06.2021 11:32:26 : EKF3 IMU0 initialised
02.06.2021 11:32:26 : EKF3 IMU0 forced reset
02.06.2021 11:32:21 : EKF3 IMU0 initial yaw alignment complete
02.06.2021 11:32:21 : EKF3 IMU0 tilt alignment complete
02.06.2021 11:32:15 : EKF3 IMU0 initialised
02.06.2021 11:32:15 : EKF3 IMU0 forced reset
02.06.2021 11:31:52 : PreArm: Radio failsafe on
02.06.2021 11:31:22 : PreArm: Radio failsafe on
02.06.2021 11:30:52 : PreArm: Radio failsafe on
02.06.2021 11:30:22 : PreArm: Radio failsafe on
02.06.2021 11:29:52 : PreArm: Radio failsafe on
02.06.2021 11:29:23 : PreArm: Radio failsafe on
02.06.2021 11:28:53 : PreArm: Radio failsafe on
02.06.2021 11:28:23 : PreArm: Radio failsafe on
02.06.2021 11:28:10 : Frame: QUAD/X
02.06.2021 11:28:10 : IMU0: fast sampling enabled 8.0kHz/1.0kHz
02.06.2021 11:28:10 : RCOut: DS300:1-4
02.06.2021 11:28:10 : MambaF405v2 0042003C 424D5004 20393954
02.06.2021 11:28:10 : ChibiOS: 08877972
02.06.2021 11:28:10 : Frame: QUAD/X
02.06.2021 11:28:10 : IMU0: fast sampling enabled 8.0kHz/1.0kHz
02.06.2021 11:28:10 : RCOut: DS300:1-4
02.06.2021 11:28:10 : MambaF405v2 0042003C 424D5004 20393954
02.06.2021 11:28:10 : ChibiOS: 08877972

This is after i'm run save all logs from spi flash

IamPete1 commented 3 years ago

I have had a constraining nan internal error on Copter 4.1.beta3 line number 234. During log download over USB, whole system seems to lock up for 1 second in every five and you end up with lots of empty log files on the SD. Sounds the same as https://github.com/ArduPilot/ardupilot/issues/15916#issuecomment-755252790 .

Some greping for constrains on that line suggest the internal error is a symptom of the system locking up and not the root cause.

xcl010 commented 3 years ago

This page is for anyone to report an Internal Error ("Internal Error:" in GCS or DataFlash messages). This allows the Dev team to be alerted, and to investigate as soon as possible.

Please enter the firmware used, vehicle type, and the Internal Error message. If you have a log, please retain it for 30 days in case a Developer would like to examine it.

Note: follow up timing on a new issue will depend on many factors, so some may not be addressed immediately.

The list of internal error numbers and meaning can be found here. 20210606183331

STRG_BAK.zip

Matek H743-Wing

Git HEAD detached at 3582b69e9c

The flight controller restarted in the air, the aircraft landed in an emergency, and the flight controller restarted repeatedly after landing. After the power is off, the flight control cannot be started, and it stops at calibrating the barometer. It can only be started by refreshing the firmware with BL using STM32CubeProgrammer, help, please! h743-wing-cash.zip

geoffreybainathsah commented 3 years ago

Hi i had an issue with mt H743 board using 4.1.0DEV arduplane fw. my airplane is a glider style airframe using -BLheli ESC with telemetry -2x FPV camera using internal relay switcher and internal OSD -RFD900 telemetry -Crossfire RX with mavlink connection

The error happend after 131min in the air, while performing AUTO mission. mission planner showed error message 'Internal error 0x1000000' followed by failsafe message I then activated RTL mode and safely landed the aircraft Luckily no reboot observed but i did have a backup 5V regulator connected to the 5V pin on the UART port which consumed 650mah during this flight

Attached is a screenshot of the error in MP and a Tlog file (incident at 91%)

2021-06-05 16-39-09.zip

note: Internal error and Failsafe could not be found on the Internal SD log file

image

rmackay9 commented 3 years ago

@geoffreybainathsah, this is most likely a hardware error because we have seen quite a few reports recently about this board and this error. We've reported it to Mateksys but it is really up to them whether they do about it. Here is a topic in the discuss forums about this: https://discuss.ardupilot.org/t/prearm-internal-errors-0x1000000-l-168-imu-reset/71658

Sekilsgs2 commented 3 years ago

18.07.2021 13:12:55 : PreArm: Internal errors 0x400 l:234 cnstring_nan 18.07.2021 13:12:25 : PreArm: Internal errors 0x400 l:234 cnstring_nan 18.07.2021 13:12:23 : EKF3 IMU0 MAG0 initial yaw alignment complete 18.07.2021 13:12:23 : EKF3 IMU0 tilt alignment complete 18.07.2021 13:12:22 : EKF3 IMU0 initialised 18.07.2021 13:12:22 : EKF3 IMU0 forced reset 18.07.2021 13:12:13 : Calibration successful 18.07.2021 13:12:12 : CRSF: RF mode 2, rate is 157Hz 18.07.2021 13:12:09 : Initialising ArduPilot 18.07.2021 13:12:07 : CRSF: RF mode 1, rate is 158Hz 18.07.2021 13:12:05 : Initialising ArduPilot 18.07.2021 13:12:00 : CRSF: RF mode 2, rate is 156Hz 18.07.2021 13:12:00 : Initialising ArduPilot 18.07.2021 13:11:55 : CRSF: RF mode 1, rate is 158Hz 18.07.2021 13:11:55 : Initialising ArduPilot 18.07.2021 13:11:50 : Initialising ArduPilot 18.07.2021 13:11:45 : Initialising ArduPilot 18.07.2021 13:11:40 : Initialising ArduPilot 18.07.2021 13:11:35 : Initialising ArduPilot 18.07.2021 13:11:35 : Disarm to allow calibration 18.07.2021 13:11:35 : Current 18.07.2021 13:11:35 : Starting calibration

After compas/mot calibration..

4.1 have many problems with ineternal errors - any fixes?

Maybe in 4.1 we have very bad optimisations when many threads cant working good with proper latency and maybe some irq's lost or have bad priority - i'm think this is main problem in 4.1 with many internal errors.

AndersonRayner commented 2 years ago

Pixhawk 4 running Copter V4.1.1 (01d1aa1e), got error Internal errors 0x100000 I:256 flow_of_ctr while out testing today. Changes from previous configurations were an auto-tune and some playing around with the EFK3 parameter to try and get the altitude estimate to be more precise (and the heading). The upgrade to V4.1.1 was completed several flying trips ago and has flown ~10 flights with the new firmware without issue.

Additionally, during the second autonomous flight of the day (log 58) the tracking started going very poorly, ended up needing to recover in STAB mode (never had this issue before). Vehicle has over 100 flights (about 24 h flight time) with no previous issue.

Logs for today (.BIN and tlogs), plus some quick flight notes and a photo of the UAV (12" props) can be found here. I have all the .BIN logs from previous flights so can provide those as required.

Thanks!

rmackay9 commented 2 years ago

@AndersonRayner, the "256" means that it must be this line that is producing the error. Is it possible that one of the COMPASS_ORIENT or AHRS_ORIENT parameters has been set to CUSTOM? I'm not sure of the exact issue but I think it is related to this.

AndersonRayner commented 2 years ago

@rmackay9 - Yes - I have COMPASS_ORIENT set to CUSTOM (100) and COMPASS_CUS_YAW set to -4 (in degrees, as per the parameter description field in Mission Planner). I have just (as a sanity check) converted my value over to radians and the error persists (even through a full power cycle). Setting the COMPASS_ORIENT back to 0 appears to have fixed the problem, however I can't flight test it (to be sure) for a couple of days.

IamPete1 commented 2 years ago

@AndersonRayner Looks like you have a UAVCAN compass, custom rotations do not work for UAVCAN if I recall correctly. Something that https://github.com/ArduPilot/ardupilot/pull/19164 will fix.

AndersonRayner commented 2 years ago

@IamPete1 - You are correct, I have a mag in the Here+ puck which is connected via CAN. I'll re-mount my compass a little more carefully so I don't need the custom corrections for now, then do the fine tuning once the PR goes through. Thanks for the quick responses!

timtuxworth commented 2 years ago

I'm getting an "Internal Errors 0x8000" on my Zealot H743 running ArduPlane 4.2.3 stable. This is happening on the bench (disarmed no GPS fix), so I'm not sure I have a log. I guess this means "Main loop stuck". The device seems to recover after reporting this, but I think I've seen it a few times on the field, always before arming, so no log. There's a screenshot here: https://www.dropbox.com/s/pcggwi3lbudregd/Screen%20Shot%202022-08-29%20at%208.51.53%20AM.png?dl=0

Hwurzburg commented 2 years ago

@timtuxworth you can get a log by setting LOG_DISARMED = 1

timtuxworth commented 2 years ago

@timtuxworth you can get a log by setting LOG_DISARMED = 1

Thanks Henry - ok enabled. Of course it doesn't happen on the next reboot. But I'll leave it on for a while and hope it pops up again.

timtuxworth commented 2 years ago

I just got another one, this time it is on a PixRacer R15 running 4.3.0 dev. It seems to be throwing 0x8000 about once every minute. See the screen shot.

Sorry it's a different FC from the one above, so LOG_DISARMED was off. I set LOG_DISARMED = 1 and rebooted but didn't see the error again.

This was "on the bench" powered by USB, happened while downloading log files. Screen Shot 2022-09-01 at 9 12 53 PM Screen Shot 2022-09-01 at 9 13 04 PM

KTechDev4213 commented 2 years ago

Recieved the following on a 3dr solo:

9/2/2022 2:26:45 PM : PreArm: Internal errors (0x800) 9/2/2022 2:26:45 PM : PreArm: Compasses inconsistent 9/2/2022 2:26:43 PM : EKF3 IMU1 is using GPS 9/2/2022 2:26:43 PM : EKF3 IMU0 is using GPS 9/2/2022 2:26:17 PM : EKF3 IMU1 origin set 9/2/2022 2:26:17 PM : EKF3 IMU0 origin set 9/2/2022 2:26:16 PM : u-blox 1 HW: 00070000 SW: 1.00 (59843) 9/2/2022 2:26:15 PM : PreArm: Internal errors (0x800) 9/2/2022 2:26:15 PM : PreArm: EKF starting GPS checks 9/2/2022 2:26:15 PM : PreArm: Compasses inconsistent 9/2/2022 2:26:13 PM : GPS: u-blox 1 saving config 9/2/2022 2:26:10 PM : EKF3 IMU1 tilt alignment complete 9/2/2022 2:26:10 PM : EKF3 IMU0 tilt alignment complete 9/2/2022 2:26:07 PM : EKF3 IMU1 initial yaw alignment complete 9/2/2022 2:26:07 PM : EKF3 IMU0 initial yaw alignment complete 9/2/2022 2:26:07 PM : EKF3 IMU1 initialised 9/2/2022 2:26:07 PM : EKF3 IMU0 initialised 9/2/2022 2:26:06 PM : PreArm: Internal errors (0x800) 9/2/2022 2:26:06 PM : PreArm: EKF starting GPS checks 9/2/2022 2:26:06 PM : PreArm: Compasses inconsistent

JAEBOV commented 2 years ago

Currently having an issue with an internal error message on a Arduplane 4.2.2 ( 693c9d91 ) system running on a Cube orange on a ADS-B standard Carrier Board from cubepilot.

20220928_183004

Anybody else having a similar issue at the moment ? Or having an idea how to track the cause of it ?

One more thing we saw is that the raw_temp output of this cube does not show 0.0 all the time, but not sure if it is related tot this error message.

hendjoshsr71 commented 2 years ago

Please provide a log if possible.

Regarding raw_temp please update Mission Planner to the latest beta that has been fixed.

JAEBOV commented 2 years ago

Please provide a log if possible.

Regarding raw_temp please update Mission Planner to the latest beta that has been fixed.

First of all thanks for the fast reply and your time.

Log of the same error:

https://www.dropbox.com/s/1izax4zvzdlh4ev/Upload_00000113.BIN?dl=0

Regarding Mission Planner, we will update it good to know the reason for the raw_temp.

timtuxworth commented 2 years ago

I continue to see Internal Errors 8000 while downloading logs on the QioTek Zealot H743. Because I am not armed I don't have a log. I tried turning on log while disarmed and I don't think I have seen the error when I do that. Happened again today on 4.3.1.

dl2ecc commented 2 years ago

I built an X8 Quad with Arducopter 4.2.3 During AUTOTUNE my Copter changed mode into LAND ( failsafe) and landed. I got an internal error 0X100000 921: flow_of_ctr I'm looking forward to your anwers. Many Thanks Wolfgang

samguru19 commented 1 year ago

I faced an internal error . I am having a hexacopter with arducopter 4.2 and here 3 gps. The setup was working till yesterday but now I am seeing this error.

image image
alariq commented 1 year ago

Matek F743 Slim. Upon boot:

12/3/2022 5:07:49 PM :  FICSR4196355 MM0 MC0 IE0 IEC0 TN:log_
12/3/2022 5:07:49 PM : WDG: T-3 SL0 FL217 FT3 FA30030768 FTP59 FLR80B3691
12/3/2022 5:07:39 PM :  FICSR4196355 MM0 MC0 IE0 IEC0 TN:log_
12/3/2022 5:07:39 PM : WDG: T-3 SL0 FL217 FT3 FA30030768 FTP59 FLR80B3691
12/3/2022 5:07:29 PM :  FICSR4196355 MM0 MC0 IE0 IEC0 TN:log_
12/3/2022 5:07:29 PM : WDG: T-3 SL0 FL217 FT3 FA30030768 FTP59 FLR80B3691
12/3/2022 5:07:28 PM : PreArm: Internal errors 0x800 l:271 watchdog_rst
12/3/2022 5:07:28 PM : PreArm: Battery 1 below minimum arming voltage
12/3/2022 5:07:19 PM :  FICSR4196355 MM0 MC0 IE0 IEC0 TN:log_
12/3/2022 5:07:19 PM : WDG: T-3 SL0 FL217 FT3 FA30030768 FTP59 FLR80B3691
12/3/2022 5:07:09 PM :  FICSR4196355 MM0 MC0 IE0 IEC0 TN:log_
12/3/2022 5:07:09 PM : WDG: T-3 SL0 FL217 FT3 FA30030768 FTP59 FLR80B3691
12/3/2022 5:06:59 PM :  FICSR4196355 MM0 MC0 IE0 IEC0 TN:log_
12/3/2022 5:06:59 PM : WDG: T-3 SL0 FL217 FT3 FA30030768 FTP59 FLR80B3691
12/3/2022 5:06:57 PM : PreArm: Internal errors 0x800 l:271 watchdog_rst
12/3/2022 5:06:57 PM : PreArm: Battery 1 below minimum arming voltage
12/3/2022 5:06:51 PM : u-blox 1 HW: 00080000 SW: ROM CORE 3.01 (107888)
12/3/2022 5:06:49 PM : GPS 1: u-blox saving config
12/3/2022 5:06:48 PM :  FICSR4196355 MM0 MC0 IE0 IEC0 TN:log_
12/3/2022 5:06:48 PM : WDG: T-3 SL0 FL217 FT3 FA30030768 FTP59 FLR80B3691
12/3/2022 5:06:39 PM : Crash Dump incomplete, dumping what we got!
12/3/2022 5:06:39 PM : Crash Dump incomplete, dumping what we got!
12/3/2022 5:06:39 PM : Crash Dump incomplete, dumping what we got!
12/3/2022 5:06:39 PM : Crash Dump incomplete, dumping what we got!
12/3/2022 5:06:39 PM : Crash Dump incomplete, dumping what we got!
12/3/2022 5:06:39 PM : Crash Dump incomplete, dumping what we got!
12/3/2022 5:06:38 PM :  FICSR4196355 MM0 MC0 IE0 IEC0 TN:log_
12/3/2022 5:06:38 PM : WDG: T-3 SL0 FL217 FT3 FA30030768 FTP59 FLR80B3691
12/3/2022 5:06:38 PM : Frame: QUAD/X
12/3/2022 5:06:38 PM : IMU1: fast sampling enabled 2.0kHz
12/3/2022 5:06:38 PM : IMU0: fast sampling enabled 2.0kHz
12/3/2022 5:06:38 PM : RCOut: DS300:1-6 PWM:7-13
12/3/2022 5:06:38 PM : MatekH743 004B0022 31315119 31313534
12/3/2022 5:06:38 PM : ChibiOS: 66e5de0d
12/3/2022 5:06:38 PM : ArduCopter V4.3.0 (93448b71)

These lasst messages (on top) are constantly repeated. This: PreArm: Battery 1 below minimum arming voltage is fine (battery is not connected)

Looking at this: WDG: T-3 SL0 FL217 FT3 FA30030768 FTP59 FLR80B3691 thread prio is 59, so I downloaded threads.txt And here is how it looks:

ThreadsV2
ISR           PRI=255 sp=0x30000000 STACK=1312/1536
ArduCopter    PRI=182 sp=0x30000600 STACK=4064/7168
idle          PRI=  1 sp=0x3001B408 STACK=144/352
UART_RX       PRI= 60 sp=0x3003B5F8 STACK=768/1104
OTG1          PRI= 60 sp=0x3003A718 STACK=272/656
monitor       PRI=183 sp=0x30016C20 STACK=1016/1360
timer         PRI=181 sp=0x30018040 STACK=1448/1872
rcout         PRI=181 sp=0x30017730 STACK=472/848
rcin          PRI=177 sp=0x300171A8 STACK=488/1360
io            PRI= 58 sp=0x30016298 STACK=1536/2384
storage       PRI= 59 sp=0x30017AB8 STACK=848/1360
UART7         PRI= 60 sp=0x30039968 STACK=200/656
UART1         PRI= 60 sp=0x30038088 STACK=248/656
UART2         PRI= 60 sp=0x300375A8 STACK=272/656
UART3         PRI= 60 sp=0x30036AC8 STACK=368/656
OTG2          PRI= 60 sp=0x30035BE8 STACK=368/656
I2C0          PRI=176 sp=0x300349A8 STACK=848/1360
UART6         PRI= 60 sp=0x30033420 STACK=264/656
OSD           PRI= 59 sp=0x3002F8F8 STACK=920/1616
log_io        PRI= 59 sp=0x30030100 STACK=952/1912
I2C1          PRI=176 sp=0x30032398 STACK=904/1360
SPI1          PRI=181 sp=0x3002DAA8 STACK=784/1360
SPI4          PRI=181 sp=0x3002D518 STACK=784/1360
Scripting     PRI=  2 sp=0x30027770 STACK=16876/17744
FTP           PRI= 58 sp=0x2001EBE0 STACK=1008/2896

So it could be storage, OSD or log_io thread. Two of them are related to writing data and taking in account that crash dump could not be fully written is seems that something may be wrong with writing data. Though I do not have enough knowledge about ardupilot to make further investigation. Any thoughts?

vandanadevatkal22 commented 1 year ago

Hi, I have got an error message saying "PreArm: Internal errors 0x10000000 I:363"

Attaching the image for reference err

Any help would be appreciated. Thanks in advance

awong97 commented 1 year ago

Hi, Firmware: Arduplane 4.3.4 Hardware: Matek H743 WLITE Airframe: Tricopter tilt rotor

I experienced an inflight error upon transiting from FBWA to QLOITER which gave an “internal error 400” message and let to loss of control of the plane with minimal damage. After disarming, the warning “Prearm : Internal errors 0x400” continued showing until power cycling the aircraft which seemed to fix the issue. The same issue was replicated in a second flight with the same trigger condition ( FBWA to QLOITER transit)

Any help would be greatly appreciated

timtuxworth commented 1 year ago

Just got this with a 4.5.0 test build - as usual while downloading logs. Luckily I had log_disarm on for something else. https://www.dropbox.com/s/h7mm2i51mz1gzmn/log_159_2023-6-12-13-48-58.bin?dl=0

timtuxworth commented 1 year ago

Another one - happened multiple (many, many) times, again while downloading some larger logs. This is the log of the download. ArduPlane 4.5.0 (master) Zealot H743

https://www.dropbox.com/s/hbi55d3n1pfhel3/log_193_2023-6-17-17-12-56.bin?dl=0

harelevy1 commented 1 year ago

After a flight of about 3 minutes I get this message 'Internal error 0x1000000' Both in manual flight and automatic flight The flight controller is a cube blue with an AUTOPILOT board

After receiving the message, the drone enters FAIL SAFE and I cannot continue the mission, it should be noted that it continues to fly normally after receiving the error

I use a cube blue but there is a version of an cube orange on it Version 4.1.5 copter cube orange

Platform

[ ] Copter

Airframe type normal copter

Hardware type Pixhawk, Cube blue