Copter: respect RC input whle in auto mode

[AndKe]

As a safety feature, any significant stick input during AUTO , RTL, or GUIDED mode, should be respected by the APM.

[rmackay9]

Does that mean that we should listen to the user input while in Auto mode? Should we switch out of auto mode when we get user input, for eample to Loiter? At the moment if hte pilot wants to retake control he/she must switch to another flight mode. I should add that there are some people who argue that we should go further and remove the small amounts of input that we do currently accept from the pilot including yaw and roll/pitch during landing.

[AndKe]

While I don't know who would remove pilots authority, for arguments sake, this "mixing" could be optional. The reason for this request, is partially based on my current CAA approved OM/POH. The relevant part of which (multirotor), today, uses mikrokopter hardware and ~95% MK software. I have pretty good dialog with the correct people in CAA (Norwegian: Luftfartstilsynet), which is well developed the past 3-4 years when it comes to evaluating and approving VLOS/BLOS UAS for commercial use.

Letting the pilot always override autopilot is considered a good practice, and when it comes to BLOS, it's even an requirement, that operator(pilot) can take over at any time. Yes, any time - would not exclude using switch, as it does not exclude the delay when flying using IP connection over cellular network. Now let's look at commercial 737 Autopilot, AP will let you do small change in attitude, tempory "mixing" - if pilot input more than, lets say 20%, then AP disengages (trigging "AP.disc" warning). This is considered a safety feature, minimizing the workload on pilot in case of sudden change is needed.

On the other hand, 767 have a switch, close to hand, needed to disengage AP, so there may not be a perfect answer to what's best.

However: keeping a non-professional user with some authority by RC control at all times, would be the right choice, because people can do errors, in WP altitude, or engange RTL while they have a obstacle between them and the aircraft.

I am not asking for a AP disconnect/mode change, but mixing RC input with AP's input (while RC input is beyond deadzone) so user can quickly increase/decrease altitude, or avoid obstacle.

So I think mixing of RC input is a great safety feature. And the way autoland works, accepting input - is great. You can correct the landing place while descending, and autoland land on a tight spot. In fact, I think that autoland, should listen even more, and disengage (switch to loiter as the closest chice ?) if user hits the throttle.

Anyway - this input is based on what i know CAA likes to see in OM/POH which are a significant part of applications for commercial operations.; that the operator can at any time, as efficiently as possible, control the aircraft.

Thanks for reading - I am just trying to help, I realize it got long.

[emilecastelnuovo]

+1 This a must to get any certification for UAV. Also in Italy the law states that the pilot must alway be in control of the aircraft so that totally autonomous flight is not permitted. This would be a nice security feature.

[msk7-ripe]

@AndKe Thank you for that great post. Yes, I also think UAV must always have control from pilot. You mentioned B737 autopilot allows mixing or trimming. Could you tell how it behaves after trimming? Does it return to previous settings or are these settings corrected with that trim. I think in automatic modes AP should accept user input as correction to programmed parameters. This means correction values should be stored and added to WP data. Also these correction may be stored in WPs allowing usage of corrected route for further flights. Also I think 100% pilot input may trigger switch to loiter mode. This isn't mandatory because RC always have handy manual mode switch.

[AndKe]

@msk7-ripe trimming while AP is engaged is ok, if you enable "stabilizer trim AP cutout" - Ap will disengage. Also, you cannot enabel AP while operating trim wheel or column.
Now we are far off topic, unless you are converting a 737 to ardupilot :) I'll stop this offtopic activity now :)

[lthall]

I have some strong opinions on this issue.

Attempting to justify this with "the pilot must always be in control" is not relevant. As AndKe pointed out a pilot can take control with a switch just as easily as with the sticks. Further if the pilot is deemed to not be in control because they have to flick a switch then they are not in control when the sticks are in their central position. So we can remove any discussion of governmental requirements from this discussion.

I will attempt to explain my position. The aviation industry decides what is good practice and what is bad practise by examining incidents and looking at what could be done to avoid the incident in the future. Over the last year or so, I have been watching the crash reports with this in mind and this is the basis for my opinions. This is not to say I know best, but I would like people to apply this thought process to Arducopter when attempting to justify why something improves or reduces safety. (While I am sure we could learn things from the way controls are implemented in the latest Airliners, for the most part the Human factors are completely different and shouldn't be applied to Arducopter blindly)

I believe we need to create a system that is easy to understand and as foolproof as possible. To me this means ensuring commands from the pilot are as clearly defined as possible. Arducopter command structure should also be as simple as possible to make it easy to understand for pilots of all experience levels.

With this in mind I will go though a couple of examples.

Pilot input while flying in Auto mode. First of all we need to define under what circumstances the pilot may want to take control.

1. Complete control to avoid a crash or abort the mission.
2. Yaw control to direct a camera.
3. Roll or pitch control to change the track slightly on the way to the next waypoint, maybe to avoid an obstacle via FPV. Then we need to decide under what circumstances we want to let the pilot take control. I would say that 1 is a must have feature, 2 is a nice to have feature, and we can do without 3 because it adds significant complexity to the system and doesn't provide a great deal of capability or safety of 1.

So now we are into the specifics. Lets consider taking control to avoid a crash or abort the mission. We need to decide how to handle this case. To do this we need to look at why the pilot might want to take control during an Auto mission:

1. no particular reason,
2. avoid collision with obstacle,
3. avoid collision with the ground,
4. altitude error caused by bad altitude estimate,
5. position error caused by band position estimate,
6. loss of telemetry link,
7. loss of radio link,
8. loss of gps,
9. low battery alarm,
10. fence breach.

So how do we handle all these things (I am sure I have missed a couple).

So lets look at the possibility that we "respect RC input while in auto mode". Now we need to decide how to interpret this input. In order of proximity to Auto we could use Loiter, Alt_Hold, or Stabilize (and Sport and Acro). Now we can detect 6 to 10 and intelligently pick a flight mode for the pilot. But what about 1 to 5.

1. Loiter is a good choice,
2. Loiter isn't bad as it will stop the copter but may not offer an aggressive enough control to avoid all collisions.
3. We could use Alt_Hold or just the throttle input to change the altitude.
4. Loiter, and Alt_Hold won't work any better that Auto, so Stabilize (manual throttle) it is.
5. Loiter is out so Alt_Hold or Stabilize.

So what do we do when we touch the sticks. And what is the chance that the pilot will move the sticks accidently, the answer is high! So once we decide exactly how we will interpret the pilot pushing a stick we need to look at what will happen if the pilot accidentally pushes a stick. What happens if the throttle is just out of the dead zone and the copter is at the limit of visual control? Unless we can find a clear and sensible solution to ALL of the issues above then we have taken a step back, not forward!

Ok, lets look at our current approach. If the pilot wants control they flick a flight mode switch. This approach can directly address all the issues above with the correct flight mode. This forces the pilot to make a deliberate decision and transmit clear command to Arducopter. Ok, so what about the negatives. The switch might be a little slower..... And in this case the pilot has much MORE control over the autopilot when compared to any attempt to use roll/pitch/yaw stick inputs. Ok, so what are the risks. The pilot might accidently switch to another flight mode, and I don't think we can mitigate that risk past audio and visual cues from the autopilot and mission planner.

So I have given a half assed analysis of this human interface issue. And to be honest I started out thinking that there might be some merit to elegantly using roll/pitch/yaw input to override Auto in some way. Having went through this process (quickly and poorly) I don't think there is a sensible way of using roll/pitch/yaw inputs to override Auto.

So unless someone can clearly describe an easy to understand and robust approach that can deal with ALL the cases above then I think it would be irresponsible for us to do anything other than use the switched mode approach.

[AndKe]

@lthall I think you are overthinging the logic involved. It's not about switching to loiter. It's like: AP thinks altitude is fine, and heading is fine . RC sticks move, AP adds/reduce throttle proportionally to the sudden input. AP lean the multirotor by adding direction from pitch/roll. It's still set on it's target, still in auto mode, but it just adds input from pilot..

Anyway - no need to be against it - like I suggested, it could easily be an option. Being a switch away, is one step to far, especially when we remember that many users here are inexperienced, untrained, not routined people with great knowledge of inner workings of the APM. I've seen for myself Quad pilot freak out and put his quad into tree, as he fought against toilet-bowl spin due to bas compass - instead of switching to stabilize, he decided to fight the flypath. He had the quad for >3 months, and flew ok. In the heat of the moment, he panicked, and reduced altitude to hit trees.
Do we trust a operator NOT to be confused for 1-3 seconds when the aircraft does not respond at all in AUTO mode ?

anyway - option is the word.

[billbonney]

The point that Lenoard is making is that adding this feature (any feature), adds more code and complexity. More code and complexity reduces safety as testing is more complex, testing coverage becomes more time consuming, and analysis becomes also more complex

Making the system less complex is how to add safety.

The example described would reinforce the issue that pilots need training for

[lthall]

AndKe I think your example further demonstrated my point. Your suggestion would just make the situation you describe worse because the pilot would tend to try to fight the toilet bowl using the stiks rather than make the decision to take manual control in Stabilize.

Saying I am over thinking the logic is just silly. I have not even half thought through the logic.

And as for "RC sticks move, AP adds/reduce throttle proportionally to the sudden input. AP lean the multirotor by adding direction from pitch/roll. It's still set on it's target, still in auto mode, but it just adds input from pilot." What input does it add? Angle, acceleration, velocity, position? How do we stop the autopilot from immediately countering your input (once you decide what that input actually is)? What does the Autopilot do when you let go of the sticks? I think you need to think through this logic a little more.

[emilecastelnuovo]

Let's make it simple enough for most people to simply disengage AUTO once you give stick over a certain amount. I bet most of pilots using Arducopter are not trained enough to "not panic" in near dangerous situations. Me myself I find it hard to act on my switches without first trying the sticks... and I have flown quite a bit.

The fact is that it is not easy to simulate emergency manouvres in daily flying, and AUTO is considered by international laws a NO-GO unless you are a trained pilot piloting military drones....

The "pilot must be always in control of the aircraft" for UAVs could be interpreted in many ways, and yes, the switch is actually controlling the drone's behaviour, but it could also be interpreted like "no matter what mode you are in, your sticks should always control the aircraft" and this is my preferred.

An example is the car's "cruise control", once you press the brake, the autopilot disengages, if you want it back you press the button to set new speed. Imagine if you brake for an emergency and after you let the brake go the car starts to accelerate because you forgot to press a button to disengage....

The idea behind this is that you have to rely on what people do instinctively not what they have to think about.

My 2 cents.

[AndKe]

@emilecastelnuovo mybe that was your 2 cents, but very neat solution, by disengage auto, should we go for switching to loiter ? (in case strong wind, so it do not blow away ?)

Anyway, in case of manual input, pausing mission, switching to loiter, is a good, low-tech solution that do not add lots of complexity to the logic.
Maybe it could even auto resume mission if sticks are left in neutral position for a few sec ? or would that be bad ?

Anyway; +1 for your suggestion.

[msk7-ripe]

I'm interested in this feature for particular purpose. I use drones for filming and there is need of auto mode to film take after take with same route. A feature allowing in-flight correction of altitude and position of programmed route would be great for this application. Also filming expects well trained pilot and drone camera operator.

[lthall]

Sorry emilecastelnuovo, "AUTO is considered by international laws a NO-GO unless you are a trained pilot piloting military drones" I have to call shenanigans on that one. I fit into the "trained pilot piloting military drones" category (more trainer than trained) and I have not heard any such thing.

This isn't cruise control. We could make a switch activated cruise control (hold the roll, pitch angles) that was deactivated by moving the sticks. That would make sense.... (at least without an in depth analysis)

So I think we have established that using the sticks to override Auto mode can't be used as a safety feature because we can't choose the correct mode unless we always switch to stabilize (unless that is what you are suggesting). If this is what you are suggesting, we need to weigh this up next to accidently moving one of the sticks during an auto mission (along with the issues below).

Ok so AndKe and msk7-ripe. The switch to Loiter if you move the sticks and the resume to Auto in 2 seconds. This covers most of the basic problems that could be encountered. (but I have only half assed thought through the issues). Now we need to understand where this fits into the brief analysis I gave above.

To do this we need to look at why the pilot might want to take control during an Auto mission:

1. no particular reason (camera operator wants to for example),
2. avoid collision with obstacle,
3. avoid collision with the ground.

All these things assume the autopilot operating at 100%.

So this could be used to temporarily stop a poorly designed mission or do some photographic excursion half way through a mission.

So now the problems:

1. So if you apply throttle up or down, does it continue on with the mission and just change the altitude or does it stop in Loiter and change altitude?
2. What do we do during land where we currently need to have the throttle at minimum to disarm the copter.?
3. So if you want to do a photographic excursion and film an object for more than 2 seconds what do you do? Do you just wiggle the sticks until you are finished?

I am sure there are more but the big one is:

How do we handle the situation where someone goes on a Auto flight and decides that the mission is wrong (lets say after only about 5 seconds). They fly back using the sticks to override the copter. They bring the copter back in front of themselves and let go of their sticks as they prepare to land. 2 seconds later Auto take control again and heads off towards the first waypoint. This takes the copter directly into the face of the young boy that has walked over to watch, blinding him in both eyes.

Randy and I have to say sorry!

I am trying to convey the importance of fully working through the issues of how we design the human interface and how important human factors are in making our decisions. It sounds reasonable, even attractive, to let the simple movement of a stick change from Auto to Loiter. But the dangers associated with it don't justify the very small benefit.

A much better option would to provide a Ch7/8 option to pause Auto. This would let the drone camera operator pause for as long as they liked (with a stationary copter), then continue the mission exactly when they wanted to. Simple, obvious, but still with an element of risk if the switch is accidentally switched back......... An improvement over the alternative.

The questions I pose are sensible and clear. We need to answer these questions before we can even consider putting this on the list of features to implement.

[proficnc]

Just a suggestion, if you want instant take over into Alt Hold, add a switch that holds the copter into your chosen mode. Place this switch on the back of your transmitter so you have to keep it held down for auto to work. If yo release the button, it will switch into Alt hold.
No code changes needed for this, add it to your transmitter mode switch logic..

Existing code for this works well.

[proficnc]

The other thing to keep in mind is that just adding input to an autopilot from the pilots seat assumes you have full situational awareness, which may not be the case in a copter. Switching to Althold to abort a mission should give you time to gather your thoughts and recover, if not, why not. Please plan your flights, if you are doing commercial work, consider the regulations, give room for error in your flight, and have a spotter (as required) The APM is 1 part of a good UAS platform. Safety is a chain, it is your responsibility to design your system to cope with your risks.

It is very simple to bypass the APMs Auto mode into Alt Hold or whatever mode you want, it is up to you to determine which mode and method suits your application.

Setup your RX failsafe wisely, setup your mode switch wisely.

If users are struggling in emergency situations, then the solution normally is Training. As pilots of real aircraft we practice for emergencies, and we set the aircraft up as such.

Though I understand the logic behind wanting to add this feature, 'I' don't think that this is the correct direction for APM.

[AndKe]

@lthall

So now the problems:

1. So if you apply throttle up or down, does it continue on with the mission and just change the altitude or does it stop in Loiter and change altitude?

For consistent input handling, it would be best if it stopped, until sticks released for 2 sec, then continue.

1. What do we do during land where we currently need to have the throttle at minimum to disarm the copter.?

This is changed in latest firmware RC, disarming after landing is supposed to happen anyway. - I think.

1. So if you want to do a photographic excursion and film an object for more than 2 seconds what do you do? This would be a planned stop, expecting operator to switch to loiter (or other mode) during this stop, - the "input during AUTO,RTL,LAND or GUIDED is primarily for use in emergency/unplanned situation.

[rmackay9]

I'm generally staying out of this discussion but for the question, "What do we do during land where we currently need to have the throttle at minimum to disarm the copter?". It is still required to have the throttle at zero to cause it to disarm at the end of the mission. I think if we have confidence that the land detector does not have false positives during landing then we could remove the requirement for the pilot's throttle to be at zero. I've never seen a false positive so I suspect it's possible to remove the pilot's throttle requirement.

[AndKe]

@rmackay9 I agree, with all the other requirements before disarm in LAND mode are enough.

Even if we still require throttle to be all way down, this is not really a problem, the land procedure could be unchanged when it comes to disarming, just react if throttle suddenly got increased - that would mean to stop descend, and switch to loiter, until stick released (and throttle lowered to below hover, again.) - then the logic would be for all auto modes, "while input - switch to loiter, temporarily"

[lthall]

I was thinking along similar lines to Randy. In fact removing the need for the throttle to be a the minimum setting maybe a safety improvement on it's own.

So we are getting somewhere in this discussion! Now if we can get around the major safety issue we might have something worth considering. It is definitely an attractive idea, now can we make it safe and practical?

Personally I still can't go past the humble switch to pause Auto and I also agree with Proficnc.

[msk7-ripe]

@lthall Thank you first for pensive analysis. I'd like to add few words about filming. As usual during filming we are at least three: pilot, camera operator and assistant. All flights are rehearsed some times and we didn't ever experience a case when operator wanted to stop mission to film "that pretty thing", but when you stand on earth you cannot exactly imagine picture from above. That's why we have to rehearse, watch video and correct mission. In filming application there's no sense of switching to other mode on any pilot input because switching ruins the paramount flight parameter for filming -- smoothness of movement. As for me, I'd like using input for path correction: instead of correcting mission by mouse and rehearsing more and more times, adjust height and position in flight and save a WP. Also this correction shouldn't be strong enough to take drone back to launch place like in your "two eyes blinding example".

I think method could make filming process more effective. First set a few WPs, set slower speed and rehearse, correcting flight path and setting additional WPs in flight.

As for safety, I always think safety first. Eg I don't think arming in any mode is safe enough etc. But you know safety is 1 divide by convenience and vice versa. We have to keep balance. Drone pilot should have enough brain to know drone is not a toy, it can be and it is dangerous.

Also I appreciate idea of pausing missions. It may work like this: if AUTO is engaged first time after arming, mission starts from beginning, if AUTO is switched off during mission, mission pointer is saved and when AUTO is switched on again mission continues. On disarm mission pointer rewinds to beginning.

[proficnc]

Due to this being a "Practical" function for inflight filming, rather than Safety, how is this for an Idea?

A new mode, Assisted Mission..... It would be a semi auto mode that allows nudging...

That way Auto can stay intact....

[rmackay9]

If it's about adjusting the mission path in flight, maybe it's something that should be done through the GCS and not the TX. At the moment you cannot update waypoints in flight but that's in the plans for the future (part of the AP_Mission library). Maybe the GCS interface needs to be improved so you can modify the path from there.

[rmackay9]

Oops, sorry, accidentally pushed the "close and comment" button.

[rmackay9]

AC3.2 now "resumes" the mission if you stop it. I think this means that if you want to do filming during a mission the pilot can switch it into Loiter mode and fly around taking pictures. Then when he/she returns it to Auto mode it will continue. I think based on this we should close this issue. I don't generally think that we should accept pilot input during auto modes.

[AndKe]

When it comes to AUTO, I agree, - maybe that should be an option ? - anyway - I agree on closing this issue.