ArduPilot / ardupilot

ArduPlane, ArduCopter, ArduRover, ArduSub source
http://ardupilot.org/
GNU General Public License v3.0
11.08k stars 17.64k forks source link

Parameters reset to default values #13134

Closed dazza-aus closed 3 years ago

dazza-aus commented 4 years ago

I wasn't going to report this as it's only happened once and it's never repeated. However, another user has reported the same issue on the discussion list and it is happening to him multiple times. James P suggested I report the issue. His post is at https://discuss.ardupilot.org/t/pixhawk-2-1-randomly-reset-parameters/50475

The other user is lakhe85 Son Pham and he was using Ardupilot on a VTOL plane.

I was flying a Skyjib coax octocopter on Arducopter 3.6.12 running a Cube black when it happened to me. Tridge was actually connected to the copter at the time and noticed it had default values which he corrected by reloading the parameters from a file. He has logs from the day. The date was Mon/Tue 18/19 2019.

auturgy commented 4 years ago

Dapo link: https://discuss.ardupilot.org/t/pixhawk-2-1-randomly-reset-parameters/50475/8

Regards,

James

On 31 Dec 2019, at 9:00 am, dazza-aus notifications@github.com wrote:

 I wasn't going to report this as it's only happened once and it's never repeated. However, another use has reported the same issue on the discussion list and it is happening to him multiple times. James P suggested I report the issue.

The other user is lakhe85 Son Pham and he was using Ardupilot on a VTOL plane.

I was flying a Skyjib coax octocopter on Arducopter 3.6.12 running a Cube black when it happened to me. Tridge was actually connected to the copter at the time and noticed it had default values which he corrected by reloading the parameters from a file. He has logs from the day. The date was Mon/Tue 18/19 2019.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or unsubscribe.

rmackay9 commented 4 years ago

I've also had this happen to me on a Pixhawk1 on my trusty IRIS about 4 months ago using a firmware version that was somewhere between 3.6 and 4.0. It also only happened to me once so I was quite surprised.

kalhe85 commented 4 years ago

I am lakhe85 on that post. Thank you so much for your support! This is a VTOL nimbus that our company purchased from Foxtech (https://www.foxtechfpv.com/foxtech-nimbus-vtol-v2.html), A RTK grey combo with Pixhawk 2.1 Standard Set with Here GNSS, firstly ran the 3.9.8 arduplane firmware. Here is the flight history of the plane: She had a crash in her third long-time flight due to battery running out (we had made some flight tests before using her for long-time missions), had some damage in the main fuselage, broke an airspeed and an ubec for mapping camera. We fixed everything, she was back to work like a charm and flied another 10 long-time missions (20-40 km distance per mission) during 2 months before the parameters reset problem happened the first time. I decided to update the firmware to 4.0.1 and reload the parameters (we did not change the parameters much from defaults of Foxtech, only the failsafe time and landing altitude that I could remember. This is the parameter file: https://1drv.ms/u/s!AjRNNKW2S9ZBlZAqR3ReA5m2aLpsFQ?e=GVlrCq) She continued flying normally another 12 missions in 2 days (about 25 km total distance per mission) before it happened again. I helped the operation team to reload the parameters, and she flied great for another 10 missions before coming back to company for regular checking and maintenance. That time, I started this discussion to find out the problem. Last week, she was out again for business. However, this time the problem happened like one or two time everyday during 4 days (we flied about 6~7 missions per day). The parameter reset just stopped for no reason two days ago when she flied 8 missions without any problem. I don’t know if the flight control reset problem might be a hardware problem due to her first crash or not. She is actually coming back to me now for maintenance and I will do additional checks on the plane if you guys need. Best regards,

mello73 commented 4 years ago

Hi, I have 3.9.8 firmware with pixracer V1 (+subs) with vtol config. After more tests, I lost some time the vtol parameters (reset to default) I don’t understand what’s happened. Best Regards,

rmackay9 commented 4 years ago

There's another report of this issue from a Copter-4.0.1-rc2 beta testers on this discussion: https://discuss.ardupilot.org/t/copter-4-0-1-rc2-available-for-beta-testing-critical-fix-included/50940/12

kkacz commented 4 years ago

This just happened to me. Brand new CubeBlack AC 4.0.2. Set all the parameters then powered off to eat some lunch and when I returned everything was reset.

rmackay9 commented 4 years ago

Here's another report of the issue this time with a CubeOrange: https://discuss.ardupilot.org/t/cube-orange-randomly-reset-parameter-to-default/52588/5

aHiroshiTsukamoto commented 4 years ago

Hi, I report a similar problem.

Yesterday and today, Parameter reset caused 3 times while drone experiments with Cube Black + Ardupilot. 2 times with copter4.0.0, 1 time with copter4.0.2.

Experiment: Take-off point is A and landing point is B. Cube power off at B. Take drone by car from B to A (it takes about 10 minutes). Cube power on at A, start the mission. Drone arrive at B in about 3 minutes.

1st reset: Ardupilot 4.0.0 After hovering for about 5 minutes with Loiter at B, cube power off. The experiment was performed 2 times and was successful. Drone's battery was replaced before the next experiment. Then, when the power was turned on at A, parameter reset occurred. 2nd reset: Ardupilot 4.0.0 After 1st reset, we wrote parameters to cube. And we waited about 1 hour because drone frame maintenance. When cube power was turned on at B, parameter reset occurred. 3rd reset: Ardupilot 4.0.2 After hovering for about 5 minutes with Loiter at B, cube power off. The drone carried more carefully compared to the 1st reset. The experiment was performed 2 times and was successful. Drone's battery was replaced before the next experiment. Then, when the power was turned on at A, parameter reset occurred.

After the 2nd reset, we wrote parameters to cube, and the power was turned on and off several times, but no parameter reset occurred. After the 3rd reset, we wrote parameters to cube, and the power on -> mission -> power off were repeated 10 times at B, but no parameter reset occurred.

auturgy commented 4 years ago

Related https://github.com/ArduPilot/ardupilot/pull/13621

rmackay9 commented 4 years ago

We have another report in this discussion:https://discuss.ardupilot.org/t/all-parameters-set-to-default-by-themselves/52983. This time it's with a CubeBlack on a "Kore" board.

pompecukor commented 4 years ago

Happened to us today. CubeOrange and "Kore" board. Flew once, land. All was fine. Then wanted to fly again and all the parameters where reset to default. 4.0.3 RC1

pompecukor commented 4 years ago

@rmackay9
Forgot to say, there was no firmware update involved. Just a re-power. Nothing was change. Parameters were not even accessed. As soon as we powered up, it was saying frame type not selected and so on. Has anyone reported it happening mid flight?

rmackay9 commented 4 years ago

@pompecukor, thanks for the extra info. No, there haven't been any reports of this happening in mid-flight. From my understanding of the problem I don't think a mid-flight loss of parameters is possible.

The "Kore" carrier board seems to be particularly vulnerable to this issue. We think it can happen with almost any hardware but about 80% of the reports are from users of the Kore carrier board.

pompecukor commented 4 years ago

@rmackay9 Thank you Wonder if spectre know about his yet...

rmackay9 commented 4 years ago

@pompecukor, yes, they do and I think they are helping in the investigation..

proficnc commented 4 years ago

May I ask what power you are running? 12s?

kkacz commented 4 years ago

Just adding my failure was on a kore board as well. Cubeblack

On Tue, Mar 3, 2020, 9:05 PM Philip notifications@github.com wrote:

May I ask what power you are running? 12s?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/ArduPilot/ardupilot/issues/13134?email_source=notifications&email_token=AHJOGQQ7QA26IUI5D7EVAT3RFWZONA5CNFSM4KBQQGBKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOENV7OMA#issuecomment-594278192, or unsubscribe https://github.com/notifications/unsubscribe-auth/AHJOGQR3CLP5MRKD5ATCVXTRFWZONANCNFSM4KBQQGBA .

pompecukor commented 4 years ago

May I ask what power you are running? 12s?

just 6S.

rmackay9 commented 4 years ago

So as you may have heard, @tridge with help from UAV Exploration (an ArduPilot Partner) has gotten to the bottom of the parameter reset issue and the fix will be included in Copter-4.0.4 and Plane-4.0.6.

My understanding is the issue is that while the bootloader is active, one of the communication lines to the FRAM is "floating" which, in very rare cases, can lead to garbage data being sent into it which can cause a reset.

The solution is to update the bootloader so we've created a wiki page here which shows how you can do this in case you want to do this before the official release go out. We would appreciate some extra testing so if you give it a try please report back on how it goes. It might be best to reply on this discussion.

Note that the bootloader is a sensitive bit of code and there is a small possibility that you could "brick" your board.

bduva002 commented 4 years ago

I found this post because it hppend to me to with arduplane 4.0.5. All I did was download some log files then come back the next day. I will go to arduplane 4.0.6 Thanks for the post.

bduva002 commented 4 years ago

Update my cube orange just reset all prams back to default again. Here is the bad part after updating the prams with my back up files I rebooted the plane. Turned the plane back on and the prams had reset to default for a third time. I happened after downloading log files with arduplane 4.0.5. I hope 4.0.6 comes out soon.

rmackay9 commented 4 years ago

@bduva002,

We know of at least two possible causes for parameter resets. On the CubeOrange the most likely is the bootloader and we have instructions for updating it here: https://ardupilot.org/plane/docs/common-bootloader-update.html.

There is also another small fix included in Plane-4.0.6 (now in beta) that should reduce the chance of it happening.

bduva002 commented 4 years ago

Thanks for the reply @rmackay9 . Is this boot loaded update only for copter? I forgot to mention I am running plane. I don't see the option to update the bootloader in MP. I am running MP 1.3.70

rmackay9 commented 4 years ago

@bduva002, the new bootloader should be included in the Plane-4.0.6 firmware as well. The "Bootloader Update" button should appear for Plane as well but you need to be connected to the autopilot at the time. image

bduva002 commented 4 years ago

Okay do I have to flash plane 4.0.6 first then update the boot loader?

I will update my MP too

I just tried it with the cube orange connected. I just see the you cannot load firmware..... Message but there is no boot loader update button.

bduva002 commented 4 years ago

Okay that was it I needed to update my MP! Thanks for the help @rmackay9

ProfessionalTrash commented 4 years ago

Just had this happen to me yesterday. clipped the ground with the quadcopter, came back and landed to re-strap the batteries, when I plugged it back in, it wouldn't arm, because everything was reset. Using a holybro pixhawk 4 running arducopter 4.0.3.

rmackay9 commented 4 years ago

@ProfessionalTrash, thanks for the report. Had the bootloader been updated to the version that comes with Copter-4.0.4 (or higher)? If not then it is probably caused by the known issue with the bootloader. The wiki has instructions including a video here.

ProfessionalTrash commented 4 years ago

@ProfessionalTrash, thanks for the report. Had the bootloader been updated to the version that comes with Copter-4.0.4 (or higher)? If not then it is probably caused by the known issue with the bootloader. The wiki has instructions including a video here.

Thanks for the info. No, I don't thing 4.0.4 had been released at the time when I set this fc up, so it probably isn't using that bootloader.

ProfessionalTrash commented 4 years ago

@rmackay9 I updated mission planner but it still only shows options for up to 4.0.3 on arducopter (up to 4.0.5 on arduplane). Is there not yet an official/stable release for 4.0.4 arducopter?

EDIT: I tried to flash the bootloader on a cube black that I am currently building so I can avoid this issue in the first place, (4.0.3 OFFICIAL) and it said bootloader flashed, however now it seems that it is stuck in bootloader mode (solid red led and flashing red led on the cube). Any help?

EDIT 2: Apparently applying external power (non-usb) got it out of the "bootloader loop," so that part is fixed now at least.

Yatin-Khurana commented 4 years ago

ArduCopter V4.0.3 on CubeBlack, Kore Carrier Board, With Herelink and Parameter Reset Occurred thrice in this week., will try updating the bootloader.

stevenyoung17 commented 4 years ago

Hi Kalhe85 From your earlier post, is this the edited parameter file or the original Foxtech one? I'm after the foxtech version with failsafes - For Nimbus V2. also are you running 4.0.5 with these parameters now- was ther eany changes? I decided to update the firmware to 4.0.1 and reload the parameters (we did not change the parameters much from defaults of Foxtech, only the failsafe time and landing altitude that I could remember. This is the parameter file: https://1drv.ms/u/s!AjRNNKW2S9ZBlZAqR3ReA5m2aLpsFQ?e=GVlrCq)

Kindly

Steve

AuClark commented 4 years ago

ArduCopter V4.1.0-Dev on Cube Orange, Cube Standard carrier board with Herelink GCS.

The Parameter reset was seen to only affect the Aircraft Frame Type (Param: FRAME_TYPE reset to 0 [Plus]) We are running a custom frame type in the HEXA class and had flown many successful flights prior to witnessing the reset. Our motor mix was generated using the iForce2d tool, added in AP_MotorsMatrix.cpp, and defined in AP_Motors_Class.h as # 19.

Is it possible that this is the same issue even though only one parameter was reset? Given we are running V4.1.0, should the bootloader update already have been included in this build?

rmackay9 commented 3 years ago

more cases of parameter resets on Hex CubeOrange boards with Kore carrier boards. https://discuss.cubepilot.org/t/loosing-parameters-on-cube-orange/4438/23

magicrub commented 3 years ago

Seems this initial CS fix could have been auto-prevented by using an unrelated scheme I've just recently switched to for an A_Periph hw device I made.

https://github.com/magicrub/ardupilot/tree/kraus/node_jDrones/libraries/AP_HAL_ChibiOS/hwdef/kha2-periph

I got tired of fiddling with the bootloader separate from the "main" hwdef so I created a hwdef_common.dat file that commonly defines all the hardware and then the hwdeft and hwdef-bl include it and set whatever specific things just for that build type.. and for the bootloader its just a couple lines. If all ChibiOS targets were like this then it would be easy to globally disable all hardware in the bootloader so noise on data lines like this would be prevented.

tridge commented 3 years ago

We had a report here with the new STRG_BAK system: https://discuss.cubepilot.org/t/loosing-parameters-on-cube-orange/4438/25 STRG_BAK.zip The error can be seen between 75 and 75:

--- STRG75.bak.hex      2021-01-13 09:01:19.595580571 +1100
+++ STRG76.bak.hex      2021-01-13 09:01:19.599580618 +1100
@@ -1,17 +1,17 @@
-000000 50 41 06 00 00 02 00 00 78 00 06 03 09 00 00 00
-000010 00 00 06 02 f3 03 3a 00 24 c1 15 00 04 ba 01 00
-000020 00 00 93 43 09 00 09 0c 09 00 93 83 09 00 00 00
+000000 00 00 00 00 00 00 00 00 00 00 00 00 09 00 00 00
+000010 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
+000020 00 00 00 00 00 00 00 00 09 00 00 00 00 00 00 00
 000030 00 00 93 03 0b 00 00 00 00 00 93 43 0b 00 00 00
 000040 00 00 93 83 0b 00 00 00 00 00 93 c3 0b 00 00 00
 000050 00 00 93 03 0c 00 00 00 00 00 93 03 09 00 22 01

It looks like the first 40 bytes were lost

tridge commented 3 years ago

interesting article that @peterbarker found: https://community.cypress.com/t5/Non-Volatile-RAM-F-RAM-NVSRAM/FRAM-corruption/td-p/136206?start=0&tstart=0

magicrub commented 3 years ago

Oh geez, I never knew the start-up power requirements were so strict and could cause data corruption like that. That feels like poor memory-chip design to me.

eduardofamendes commented 3 years ago

Just had the same problem with Arducopter V4.0.4

rmackay9 commented 3 years ago

@eduardofamendes, Copter-4.0.7 is out now with numerous fixes vs 4.0.4 for this issue. Afterloading 4.0.7 it might be good to update the bootloader but in either case it shouldn't happen with 4.0.7.

eduardofamendes commented 3 years ago

I am doing it right now @rmackay9. By the way, the FC is a Cube Orange. Thanks!

IamPete1 commented 3 years ago

No fresh reports, I would quite like to close unless anyone has a update?

rvb2022 commented 1 year ago

Today have this issue on Pixhack V3x, ArduCopter 4.2.3, updated bootloader. After flight drone battary was disconencted. Later on connected FC to Laptop with USB cable to download flight log. When connected to FC: warning messages came up (unsupported frame, compass calibration required, accel calibration required etc), all arameters reset to default vlues. Loaded parameters from file, but still accel clibration is requested.

rmackay9 commented 1 year ago

@rvb2022,

We think the parameter reset is caused by a issue with the eeprom chip which can lead to a reset if the power dips during power up. We've added protection for this on the boards with enough room in the eeprom to store a backup but for those boards without storage there isn't much we can do I'm afraid. Luckily the issue is relatively rare it seems.