EKF doesn't utilize optiflow well when GPS goes bad

[RickReeser]

Bug report

Arducopter 3.6.7 Copter, X4 frame Pixhawk 2.1 cube black

Lately I've been testing ways to harden UAVs against GPS-related failures, as it is a single point of failure for most systems. I've been testing optical flow as a way to retain controllability in the event of a GPS failure, with mixed success. I have a way to easily simulate GPS jamming/spoofing/etc. in real flights, and in my testing I've found four modes:

1. If the GPS fails and does not send any data at all, optiflow is used to hold relative position and fly in navigation-enabled modes. This works very well.
2. If the GPS is providing bad data (spoofing, multipathing, etc.), the EKF consumes this data and the drone drifts around. This might be a matter of tuning EKF parameters (suggestions?).
3. If I tighten the EKF innovation gate parameters and then send bad GPS data, the EKF triggers failsafe and enters the "no GPS land" mode, as if I had no optiflow at all. #9919 seems related, but I am using EKF3.
4. If the GPS is jumping around a lot, sometimes the glitch detection will reset the position estimate to the GPS location. Not a bad thing, but for some reason this causes the drone to try and correct violently in the opposite direction.

I'm submitting this as a bug report because of the last two points. The second point may just need tuning, or it might end up being a general enhancement for using local positioning for dealing with spotty GPS.

My ArduCopter is modified with the optical flow max altitude scalar height_max set to 99 instead of 3 meters for testing optiflow at higher altitudes. Also, I'm using EKF3 for these tests, but EKF2 seemed to have the same behavior, and I also have an EKF2 running on the same IMU for comparison. Here's some logs that demonstrate the above points:

"Good" failure where GPS cuts out and optiflow holds position: ekf3 optiflow jammed success.zip

"Bad" failure where the drone drifts around (first part of log), then goes crazy when GPS resets absolute position (end of log): Just before I hit Stabilize, the drone zoomed off northwards. ekf3 optiflow jammed bad.zip

Log with EKF failsafe triggering unaided Land mode (first part of log): Parameters that are different from above logs: EK3_POS_I_GATE 100, EK3_VEL_I_GATE 100, EK3_GPS_CHECK 50. Defaults are 500, 500, 100, respectively. Note that the drone knew that it was drifting forward in Land mode, but made no attempt to correct. ekf3 optiflow jamming small i gate.zip

[chobitsfan]

Hi @RickReeser Would you be able to take a look at #12482? Thank you very much

[RickReeser]

Here's two more logs on Copter 3.6.12 that demonstrate item 4. above: the drone goes crazy when the position resets.

gps jam position reset.zip