Copter: ensure hexacopter with failed motor can make safe landing

[t0nimas]

Hello,

I would like to know the APM response in fornt of an engine failure. As far as I knew, while flying an Hexa/Octo, the APM should be able to ensure the safety of the platform. However, last days, during one of my tests, I tried to fly (automatic) an hexa-rotor with one engine failure and the platform fell down (1000 feets) with any sign of APM response.

Then, once I arrived to this point, the theory that I was using an hexacopter to ensure the safety due tu redundancy is not working anymore.

This is the main reason why I am writing this post. Which is the best way to ensure safety infront of an engine failure in flight? APM code is designed to confront this kind of situations?

Which is better:

• Octocopter?
• Coaxial copters (quad,hexa,octo)...

[billbonney]

I've seen a APM hex fly with one motor out. But you do need enough power from the 5 motors to fly. It's the same with any redundancy setup. You need enough spare power from the other motors to be able to compensate the loss

[rmackay9]

A hexacopter also loses yaw control when it loses a motor. Because of the set-up of the motors it's actually not simply a quad with two extra motors.

Coaxial copters like the Y6 can fly with one motor failure. Octacopters (whether in a quad shape or spider shape) can also survive a motor failure.

We have plans to integrate some smart ESCs that will help us detect motor failures and perhaps they will be more resilient to sync failures but at some point physics takes over and there's little the software can do especially on the APM2 where the CPU is already overloaded.

I think there are some things we can do if we know a motor has failed on a hexacopter including giving up on yaw control. I've renamed this issue's subject line to make it more specific.

[rmackay9]

A blog post about a quad which can recover from a single motor failure. http://diydrones.com/profiles/blogs/onboard-quadrocopter-failsafe-flight-after-actuator-failure

[francisvierboom]

Are there two options for hex redundancy behaviour?

• give up on yaw (same as the quad), spin like a madman and maintain most of 83% (five 6ths) of original power - spinning continually compensates for pitch/roll motor imbalance. Possibly some loss would be in actually powering down the two minority motors to get yawing up to the required speed. Randy, as you say, the rotational algorithms to maintain roll and pitch control in this state must be a bit heinous, which is the special sauce in the (incredibly impressive) quad redundancy video from IDSC/Flying Machine Arena
• maintain full attitude including yaw, by detecting a motor failure somehow (possibly detected with existing sensors from a sudden unprompted roll/pitch change that's exactly along the angle of a motor?), which leaves only 66% (four 6ths) of original throttle sum - cut out the opposite motor, and fly as a quad, rather than distributing it among the three on the healthy side for roll/pitch compensation (which would cause unstable yaw). But this 'quad' may still have yaw problems because each of the remaining motors will be spinning the same way as the one next to it across the dead arm (rather than the usual quad setup of matching motor spins diagonally opposite each other). And 66% of original power is not fantastic.

I guess it's a major benefit of quoctocopters (can this catch on for X8s?) that they can lose a motor, cut one opposite motor and maintain R+P attitude with 75% of original throttle available (6 out of 8 motors) plus the efficiency bonus of no longer having two props sharing the same wash - is that right? Does the spin imbalance across the unhealthy arms still cause a yaw problem and force it to power back down to effectively 4 out of 8? Don't know enough about how yaw is controlled to assess this :S

[wjax]

This is a great feature in terms of safety Just seen the video of NAZA managing a motor loss in an hexa beautifully! I am sure we can do it too! And better!

[ghost]

Some interesting reading about solution this guy uses a Kalman filter for estimation & check if something failed I checked it already but it is out of my mathematical knowledge :) http://www.feec.vutbr.cz/EEICT/2014/sbornik/03doktorskeprojekty/03kybernetikaaautomatizace/01-xbaran10@stud.feec.vutbr.cz.pdf

[rmackay9]

I've created a patched version of AC3.2 which allows reducing the speed of one motor in case anyone wants to give it a try, this is how to use it:

1. Load one of the two firmwares below onto your Pixhawk (sorry, only quad or hexa although I could compile others) a. Hexacopter: https://dl.dropboxusercontent.com/u/3852647/AC3.2-motfail/ArduCopter-3.2-motfail-hexa.px4 b. Quadcopter: https://dl.dropboxusercontent.com/u/3852647/AC3.2-motfail/ArduCopter-3.2-motfail-quad.px4
2. Go to the full parameters list (or tree) and set these parameters: a. TUNE = 99 b. TUNE_LOW = 0 c. TUNE_HIGH = 1000 d. MOT_FAIL_NUM = 1 (the number of the motor that you want to fail)
3. Test it’s working by moving the ch6 tuning knob up and down and push the Refresh button on the MP’s full parameter list and check the MOT_FAIL_PCT is changing from 0 to 100 (0 = motor off, 100 = motor normal power)
4. Plug in the battery and test again that you can shut down the motor by putting ch6 to its lowest position

So with this it should be possible to test a motor failure in a somewhat safe manner because you’re able to reduce its power in stages to see how it reacts. So you could try reducing the power by 10%, 20%, 50% etc to see how it responds. In the simulator, the overpowered quadcopter can handle one motor’s efficiency being reduced by 45% before it flips.

The patch consists of the last two commits in my repo’s AC32-motorstop1 branch: https://github.com/rmackay9/rmackay9-ardupilot/commits/AC32-motorstop1

I’m not planning on putting this into master because I can just imagine it’ll lead to more harm than good but that’s open for debate. At least for now we have a patch that people can use on an as-needed basis.

[rmackay9]

pushed to AC3.4.

[firdel]

Interesting thread. My partner and I have to show a working hex with redundancy at a flight test for the Danish Transport Authority in aprox a month. We choose the pixhawk FC over the A2 because of the open platform. It is a demand when flying with a setup above 1800grams, with people located inside your safty zone (Danish rules), that the UAV can be moved away from a specific point, with 1 engine failed. Do i understand correctly, that with your patch rmackay9, we can demonstrate this, in flight? With yaw out of controle, is there any controle as to moving the UAV in a specific direction? Im very new to the pixhawk, and im not responsible for the patching/programming part. Its not my area. We do have a very competent man helping us. He guided me to this thread. If there is some directional controle over the UAV, with your patch, and 1 engine shut down, could this be implemented in our pixhawk to work "IRL", and not just in a test enviroment. As i read the posts here, the only controle left, in case of engine failiure, is vertical controle?

Best Regards /firdel

[ricardocrl]

Hello everyone, I'm little familiar with the ardupilot and I am trying to understand its current status and future approach on this topic: redundancy in Hexa and Octa copters in case of motor failures.

I'm working in an engineering company and we are happy to see this discussed and at least know that it is a question of time to see something implemented.

My question now is, what is the planned approach? Or no plans yet? My understanding is about the ardupilot philosophy is to make everything as hardware-independent as possible (e.g. special ESC). Would this be possible?

Thank you

[firdel]

Hi Randy. I have been flying quite some with with your motor stop simulation branch on a Hexa (https://github.com/rmackay9/rmackay9-ardupilot/commits/AC32-motorstop1), with mixed results. What I am trying to achieve with one engine out is to fly the hexa away from its motor fail location (something like 20-30 meters), and then if possible land. When testing I have mostly been in stabilize, and what happens as one engine is turned down is that I have to compensate for yaw to keep it from spinning. This results in there not being enough thrust to keep it in the air, nor to fly it 20-30 meters away.

If you could give me your 2 cent on the following questions I will be very grateful.

Which flightmode do you think would be best for the above situation? -And which would only make matters worse? Should one ignore the spin, and then do the flyaway in simple/super simple? I have currently been using 3.2 firmware with the motor fail branch, would there be any benefit in using 3.2.1 or 3.3-rc10 with the motor fail branch instead?

Regards

[lthall]

Hi all, A hex will need to reduce yaw authority to maintain roll and pitch control. So if you want to make sure your hex will survive a motor out you need to ensure you have a good roll and pitch tune. You must ensure you have plenty of additional power, your copter should be able to lift 3 times it's take off weight (not as hard as it sounds).

Things that will improve the copters ability to maintain roll and pitch performance you should keep the Rate_Yaw_Imax and MOT_YAW_HEADROOM low.

So because you should be losing your Yaw authority if you lose a motor, you should use simple mode of some flavor or simply RTL.

[Sunrik]

Which flavor of simple/super simple do you reckon would be safest to test with loiter+super simple, alt hold+super simple or stabilize+super simple?

What is considered a low value of RATE_YAW_IMAX?

Finally, I cannot find MOT_YAW_HEADROOM in the parameters list?

[rmackay9]

Linking new issue related to this one: https://github.com/diydrones/ardupilot/issues/3294

[roque-canales]

Randy, there's possible for you to share compiled 3.3.1 hexa motostop version for test it ?

[csriddel]

Randy, I think I have a real example of a motor (or ESC) fail for a hexcopter (DJI S900). I believe motor 2 (left) stopped working for whatever reason -- and it would seem the pixhawk shut down motor 1 to compensate... is that what you'd expect? Log linked below. The result was a rapid descent and crash. I'm not sure I'm seeing a loss of yaw control, just severe pitching around the longitudinal (x?) axis.

https://www.dropbox.com/s/bd9j9zv70spx3wn/2015-12-22%2012-06-24.zip?dl=0

[matheweis]

Just came across this thread while discussing ArduCopter's capabilities in terms of handling rotor failures in hex mode with another user. Just in case this will be of any help, here is the log of a flight that experienced catastrophic failure of motor 5 (front right) while running stock 3.2.1: http://eisbox.net/tmp/15-08-17_19-13-20.bin Sorry about the age, I knew the failure wasn't due to ArduCopter so I wasn't posting all about it, and I'd forgotten all about any use for the flight logs until I downloaded them recently.

[rmackay9]

I think AC-3.4 performs fairly well but perhaps we haven't done enough controlled tests to close this issue. Pushing to AC-3.5.

[RJBeckwith]

Thanks Randy. I had a motor failure once with a Hexa running a DJI Naza32 and it recovered very nicely. Turns out a motor from a previous crash had a connection problem and I didn't know it until the motor stopped in mid air. Very cool to land the thing without incident.

I would love the same capability with Arducopter.

[lthall]

There is some basic and unavoidable maths at play here. To maintiain roll and pitch athority in a hex conficuration without 1 motor and without chainging the motor mixers you give up a lot more than 1/6th the lift capacity (1/3 I believe but I don't have my calculations on me). You can get a little of that lift back but if you know what motor is out but you still have only 50% thrust (I have to double check my results).

So if you have a hex the limitations above are unavoidable and represent the best any controller can do (without making use of the dynamics of a fast yaw spin).

The only way to not loose yaw control is if the CG is offset in the right direction so if you loose a motor and you don't loose yaw control then you probably got lucky because if you lost the motor on the other side you probably would have had an unavoidable crash.

Given a properly set up hex you also need a very well tuned hex with yaw Imax and yaw headroom set low to maximise the roll and pitch athority. If you don't have this then again the copter crashes.

So while I understand the desire to have ensure a hex can make a safe landing we can't do rewrite the laws of physics and we can't build and tune every hex as we arn't selling ready to fly copters. The only thing I would say is we may be able to add some notes to the wiki on how to maximise the chances of a motor out situation on a hex. We may also be able to drop the yaw mix to zero in the case of large roll and pitch errors.

I personally recomend we close this issue.

[R-Lefebvre]

I would just like to add to this issue, that while we cannot rewrite the laws of physics, we should at least try to detect the condition and warn the operator. This is the case especially with octocopters. But I've even seen hexacopters flying normally with a motor failure. But if it's far away, the operator doesn't know. They might want to land right where they are if it's safe. If they starting trying to fly fast, it could lead to a crash even though the crippled copter could hover safely.

I know we've talked about this Leonard, just officially entering it in Github. :)

[robustini]

During a single motor failure with S900 and DJI FCU the pilot It can govern the hexa where it wants using the "Course or Home Lock" (Simple/SSimple in APM:Copter):

https://www.youtube.com/watch?v=HQ7wa5cBT_w

It's not difficult to move the drone with "S" and "SS" during yaw turn and I often shown:

https://youtu.be/RF0wnQv3aAc?t=334

So it's just a matter of code and payload, ArduPilot can and should do. I asked for a long time to have the possibility to turn on/off a motor through a switch for testing, upon request ignored... :-)

[panky0]

hi, noticed your discussion because today my hexa crashed because a self locking prop (Motor #2 "left-middle") did unlock itsself and my hexa went down (not really crashed, just went down in a more or gentle way and to the side (where the prop was missing) - only my 3d printed legs were harmed); i was flying position hold; I have a Navio2 (by emlid) and using 3.4-rc4; but i didn't do any of the recommended changes to MP settings; i use 6x turnigy 3549/6 790kv motors with 4s and ( i know at least 2 inches too big) 13inch props; - but usually my hexa flies overpowered -maybe not overpowered enough; my very raw estimation is about 400w per motor (with these oversized props) and almost 5kg AUW; here is the log - hope it helps: https://drive.google.com/file/d/0B9PqIZ8f2EDsZUhfVnEzU09VY28/view?usp=sharing (current is wrong since i have 2x16AH 4s lihv and cannot fly more than 20min at the moment) i didn't have much time to save the copter, tried to steer against the drift in direction of the lost prop - but not enough; and i didn't change flight mode; yaw didn't change drastically...

[lvale]

Adding to @robustini comments this video https://youtu.be/hFZ_d9mbVGU?t=1m20s from 1:20 gives some good leads. Stopping motors, enabling SS. (and considering that there is enough power (like @lthall mentions too)

[geofrancis]

i have had motor failures on a couple of hexacopters now and from what I have seen it wont fall out the sky but pitch in the direction of the failed motor and slowly decend, the problem is that when it pitches it accelerates and keeps picking up speed until it hits the ground.

generally it will take less damage than a quad falling vertically downwards but its not ideal if your working in an enclosed area where the things around the aircraft are far more expensive than the aircraft or there are people around.

from looking at the logs the throttle output to the failed motor always hits 100% and is held there until it crashes.

To detect a failed motor what about checking if a motor output is at 100% and des pitch and roll are out by a number of degrees in the failed motors direction

motor 1 failure would roll to the right motor 2 to the left motor 3 would be forward and to the left etc

then lower the yaw headroom and imax automatically to keep pitch and roll at all costs, you could even have parameters for failed motor that could be set this would put the machine into a fast spin in order to keep pitch and roll but it could be landed with some degree of control.

[lvale]

Assigned to @rmackay9 because he has already done some study on this.

Perhaps this PR might also be relevant https://github.com/ArduPilot/ardupilot/pull/3672, ie detecting unexpected saturation of outputs for some time and adjusting the other outputs ?

[csriddel]

We’ve lost two s900 helis from what appears to be single ESC failures. The failed ESC seems set to 100%, and the opposite ESC gets 0%. That leaves 4 motors to maintain altitude and stability and it just doesn’t seem to work. Perhaps I’ve diagnosed incorrectly. I’d be happy to forward .log and .tlog files. Steve

[gerbertkuiper]

Very interesting discussion here. I would like to add the following;

Mounting the motors tilted on the arms increases the yaw authority drastically and allows for a much lower yaw headroom (thus greater chance of survival after motor failure).

Tilting a motor 5 degrees around the arms length axis (so that the thrust reinforces the yaw momentum of the motor torque) makes that 8.7% of the thrust is working as a yaw moment on the copter's frame. 99.6% is still working to keep the copter airbone. So it does not reduce the flight time significantly.

Another advantage of motor tilt is a more stable vertical descent because props are less suffering from their own propwash.

I agree with geofrancis that in the end the best solution is that if the flight controller can not hold the desired roll and pitch angles, it should automatically put yaw headroom to zero and enter (super) simple + Return to Launch mode to safe the hexacopter.

[R-Lefebvre]

Yes, but keep in mind, that motor tilt will also result in the reverse action. Lateral airspeed will result in more or less lift on that motor, which can then make roll/pitch stabilization more difficult.

This effect has not been well characterized, but theoretically, it's worth considering. Just pointing it out.

[wjax]

Given the general poor quality of ESC all around, this item of UAV surviving a motor loss should keep all our attention. Realiability and safety should go before new features IMHO!

[R-Lefebvre]

If you use high quality ESC's, there is no issue.

[geofrancis]

i have seen expensive escs and motors fail, just less often,

[wjax]

No matter how good is. I WILL fail.

In aeronautics a single failure should never imply a catastrophic safety event.

Having an autopilot that can handle a motor failure and land with minor damage is just a crucial feature and much more important than any other.

[csriddel]

I have two logs in less than two years that seem to indicate an esc/motor failure on two DJI s900s. It's an expensive and dangerous problem.

On Nov 19, 2016, at 09:03, Jesus notifications@github.com<mailto:notifications@github.com> wrote:

No matter how good is. I WILL fail.

In aeronautics a single failure should never imply a catastrophic safety event.

Having an autopilot that can handle a motor failure and land with minor damage is just a crucial feature and much more important than any other.

You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://github.com/ArduPilot/ardupilot/issues/838#issuecomment-261718858, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AP181mOJGs73n5HmFiTOV5XqbS43zYIPks5q_w-6gaJpZM4BhVd9.

[tabascoz]

Hello, I would like to inform a crash with one motor failure. http://discuss.ardupilot.org/t/hexa-crash-one-motor-failure/13576?u=tabascoz Willing to test and help on this matter!

[peterbarker]

Here's a simple patch to simulate a motor failure in SITL: https://github.com/ArduPilot/ardupilot/compare/master...peterbarker:sim_mot_fail?expand=1

The parameter SIM_MOT_FAIL is a bitmask of motors to fail.

[AndKe]

First: @rmackay9 's test patch,(disable a motor midair) should be a standard part of ArduCopter - triggable by CH*_OPT
Any user would be more aware of actual behavior , and could test/train more easily then.

Then: We have one clear-as-day sign of motor failure: , the failed motor's RC.OU is max, the opposite motor is commanded to idle. In addition, CTUN.ThO tells us throttle demand is maxed out.

In such situation, remaining motors should be mixed so they were unaffected by the failed/ and idle motor. If 6 failes and 5 is commanded to idle, then 1,2,3,4 should be scaled up to 100% or until thrust demand is met.

[magicrub]

In most cases, throttling to 100% means you've lost the ability to control since you have no headroom for feedback.. but in this case doing 100 should probably be allowed and made an exception for in the motor lib.

On Dec 27, 2016 5:38 AM, "André Kjellstrup" notifications@github.com wrote:

First: @rmackay9 https://github.com/rmackay9 's test patch, should be a standard part of ArduCopter - triggable by CH*_OPT Any user would be more aware of actual behavior , and could test/train more easily then.

Then: We have one clear-as-day sign of motor failure: , the failed motor's RC.OU is max, the opposite motor is commanded to idle. In addition, CTUN.ThO tells us throttle demand is maxed out.

In such situation, remaining motors should be mixed so they were unaffected by the failed/ and idle motor. If 6 failes and 5 is commanded to idle, then 1,2,3,4 should be scaled up to 100% or until thrust demand is met.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/ArduPilot/ardupilot/issues/838#issuecomment-269327375, or mute the thread https://github.com/notifications/unsubscribe-auth/AEj7G-Y9VIxN-bq1x7gtZ6ZhEgFWVZnYks5rMRTlgaJpZM4BhVd9 .

[tabascoz]

It could be less than 100% enough to allow small pitch/roll movement.

[AndKe]

@magicrub @tabascoz Of course, I did not meant it like that. I am well aware of the need for control. I just assumed everybody here would understand my meaning, not take it literally. Let me rephrase; "remaining motors should be scaled up till one of them (depending on attitude) hits 100% , or required total thrust is met"

Or: to put it in another way; the post with a log @tabascoz posted a link to some days ago. illustrates my point pretty well: One motor failed, the opposite was set to idle. The vehicle drops altitude fast, yet, there is no sign of the remaining four good motors working any harder. - There's plenty more thrust to get, but it simply not used, due to the way motor throttle is mixed. It seems to me that if - in that situation, motor throttle for 1,2,3,4 were scaled up , until any of those, in any moment, reached 100% - or thrust demand was met, then the hexa would just fine.

The sharp minds here, can - as I described earlier, easily detect such situation, and make AC handle it.

[magicrub]

I was literally saying that in this case going to 100% may be OK. I'm thinking for a quad though, not a hex.

On Dec 27, 2016 11:27 AM, "André Kjellstrup" notifications@github.com wrote:

@magicrub https://github.com/magicrub @tabascoz https://github.com/tabascoz Of course, I did not meant it like that. I am well aware of the need for control. I just assumed everybody here would understand my meaning, not take it literally. To put it correctly , would be: "motors 1,2,3,4 should be scaled up till one of them (depending on attitude) hits 100% throttle requirement, or required total thrust is met"

Or: to put it in another way; the log @tabascoz https://github.com/tabascoz posted a link to some days ago. illustrates my point pretty well: One motor failed., the opposite is set to idle. The vehicle drops altitude fast, yet, there is no sign of the remaining four good motors working any harder. - There's plenty more thrust to get, but it simply not used, due to the way motor throttle is mixed. It seems to me that if - in that situation, motor throttle for 1,2,3,4 were scaled up , until any of those, in any moment, reached 100% - or thrust demand was met, then the hexa would just fine.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ArduPilot/ardupilot/issues/838#issuecomment-269371347, or mute the thread https://github.com/notifications/unsubscribe-auth/AEj7G1S6B5Tk1IlJYGVcaliRlpVQFHRRks5rMWa_gaJpZM4BhVd9 .

[tabascoz]

Randy's patch ported to Copter-3.4 branch: https://github.com/tabascoz/ardupilot/commits/Copter-3.4

[El-Mango]

https://www.youtube.com/watch?v=w2itwFJCgFQ#t=6m21s

[AndKe]

@El-Mango that's not very relevant, those quads are surrounded by cameras , and powerful image processing is telling them what to do, so they are not bothered by gyro and accelerometer limits like "real" /independent machines. Also, most quads may not have enough power , or asymmetric payload , and be unable to achieve yaw rates needed to fly that way before crashing.
-So while it's fancy, not a practical solution.

Anyway - this thread is about much simpler handling of such situations for hexa/octo multirotors. (which is simpler) A solution for a quad would be very different.

[geofrancis]

why is the yaw headroom so high by default? it sounds like that could be the cause of most of the problems of loosing controll due to motor failure? i would rather it was spinning on the spot than flying sideways.

[AndKe]

@rmackay9 What's your opinion about this ? Should't it be fairly easy to detect/handle a problem in hexa and octo, as I described above ?

[El-Mango]

@AndKe My opinion is that, even if those quads are surrounded by cameras, that doesn't effect the laws of physics... Fancy or not. The only way to keep up those things is exactly with gyros and accelerometers as we do. Spinning is a reaction, considering that the two elements of the remaining working pair of propellers are spinning in the same direction. So, you let the system spin in aim to make the two remaining propellers spin at the the speed that you need. In this condition, it's possible to stabilize the altitude. (remaining pairs are keeping up the system) Even if the system spins, you need of course to keep track of its position and orientation in every moment. At that point, applying an extra sinusoidal power, synchronized with system's spin phase, you can determine a movement of the system in the direction you want, keeping a constant altitude. Condition for the system to be stable while spinning with remaining working pairs, is that point of application of force of the props must be higher than the Center of Gravity, otherwise it would flip upside down with propellers pulling towards hearth.

[AndKe]

@El-Mango I am not going to argue with you here, it's just so much more to "promise" people with quads that already fly at 50% thrust to quickly get into the velocity needed to achieve that.

While possible - it's still a very different situation (and solution) than what's needed in this case. (see issue title)

Yes, I would love to AC do quad recovery in real life (actual useful machine with payload do that autonomously when needed) - imagine ArduPilot retrofitting Solo with such ability - that would impress a lot of people...

[El-Mango]

I just pointed out this case study because a strong theoretical solution, if proven working, would provide a more consistent support to all possible implementations, rather than empiric ones. It would be nice to have a "Recovery Mode", entered after fail detection, valid for all copters in general and verifiable with some sort of auto-tuning...

[geofrancis]

the problem el-mango is that very few machines are perfectly balanced on all 3 axis like the ones in the video, most quads would just wobble out the air if they tried to spin that fast.