RTH: Started flying backwards away from home, caught on bbox and video

[theArchLadder]

Video. RTH and fly-away happens around 1m5s. The camera is at the true home position and is looking west/north-west. Quad is heading north during poshold, turns to east as expected during ascent, then starts flying backwards towards west, away from home, out over the water! (btw: look from the beginning for beatiful poshold!) https://www.youtube.com/watch?v=1ppEUqvIGCM&t=1m5s

Blackbox log: Log 32/34 is the one in question here. I could not use blackbox_decode on this, it only decoded to log 31...? Blackbox viewer had no problem though. Would be nice to see the raw gps coordinates. https://www.dropbox.com/s/hcxawi62zerswpy/blackbox_log_2016-05-13_115326_RTH-backwards-fly-away?dl=0

Dump: https://www.dropbox.com/s/0u2do430qpgp23l/blackbox_log_2016-05-13_115326_RTH-backwards-fly-away_dump.txt?dl=0

What i see in the blackbox log:

1. navPos starts at 244, -130
2. During poshold, navPos is around 800, -800, so 5,56 meters north and 6,7 meters west of home pos, sounds about right.
3. Very beautiful and stable poshold for a couple of minutes.
4. RTH is activated at 2m3.5s, starts ascending and yawing toward home (east) as expected.
5. At 2m4.8s it starts to fly backwards, navVel[1] shows around 800cm/s before the quad has even started to bank, and this is velocity towards east, right? In that case, it's the wrong direction! At 2m6s navPos[1] shows 2300, AFAIK, that's 23 meters east of home? The quad was actually more like 15 meters west of home. navTgtPos was still 800, -800, so my quad thought it was like 30 meters east of that and started flying towards west to get back i guess.
6. After aborting the RTH i tried some quick poshold, they seemed to work, i landed at 2m50s and the log claims that i landed 10 meters east of home, but i landed just a few meters from home.

My guess on what cause this: To me it looks like the GPS pos went crazy wrong, i looked at the config to see if i could have messed something up, and there it was, HIGH_G has changed name to AIR_4G, but my setting had reverted back to default AIR_1G! I have seen this exact fly-away behavior multiple times with PEDESTRIAN a few moths ago when I did hard braking, but I never caught it on bbox. Since then i have been flying with HIGH_G and have never had any fly-aways. What is scary here is that the GPS seems to got wrong position just because the quad started ascending, there was no horizontal movement! SBAS was set to NONE.

Or the cause of this problem could be something completly different, who knows :wink:

Would be awesome if anyone could help me decode the log so we can look at the raw gps data.

[theArchLadder]

I just realized that i got a relative position error of around 30 meters or more in just 1.2 seconds... Scary.

[stronnag]

parser.h

define FLIGHT_LOG_MAX_LOGS_IN_FILE 31

set to e.g. 64 and your log 32/34 decodes (replaying in mwp now). Maybe ask Nick about the limit?

[nathantsoi]

everyone probably knows this, but arducopter and px4 both do EKF filtering and switch from gps guided mode to landing mode on failures

http://ardupilot.org/copter/docs/common-apm-navigation-extended-kalman-filter-overview.html

does inav have an ekf? do we want one? perhaps it could help in cases like this?

On Fri, May 13, 2016 at 9:50 AM stronnag notifications@github.com wrote:

parser.h

define FLIGHT_LOG_MAX_LOGS_IN_FILE 31

set to e.g. 64 and your log 32/34 decodes (replaying in mwp now). Maybe ask Nick about the limit?

— You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub https://github.com/iNavFlight/inav/issues/206#issuecomment-219098089

[theArchLadder]

@stronnag Thank you very much! I had 18 sats for a long time, then this happend:

1. 0 sats for ~200ms
2. 15 sats for ~100ms, 25 meter instant position jump to east.
3. 0 sats for ~100ms, 15 meter instant position jump to west.
4. 15 sats, stable after this but position looks to be ~20-30 meters to far to east... I need to use a graphical tool to see more clearly what happens, guess it's time to use mwp! @stronnag feel free to post one of those fancy video/image you make...

[stronnag]

you missed out the sudo.

Anyway, I had a wonderful 15 -18 sats with consistent low HDOP for 45minutes air time (and gcc 6.1.1, which I'm begining to like a lot). Sh*t happens.

[theArchLadder]

@stronnag I know, i should stop being lazy and do it myself! I have also had wonderful GPS performance since the sbas fix, stable sat count, up to 20 of them! It was just this weird incident.

You fly in HIGH_G/AIR_4G right? I'm curious if my AIR_1G could have something to do with it since i did have very similar issues with PEDESTRIAN, all those fly-aways happend during hard braking though.

I agree that shit happens, but it would be wonderful if inav could deal with that, fly-aways are scary!

Why use a finished program when one can hack together something in processing? :stuck_out_tongue_winking_eye: One green dot for every gps position, the white line shows that instant jump in position... Looks like almost a 40 meter jump in position. Hmm, i think i landed very close from where i took off (center of circle), so i guess the GPS never really recovered

[stronnag]

I've been flying in AIR_1G recently, seems a good compromise to me, but then I'm on a 420mm quad or 600mm tri, so probably more sedate flying. I will generate a video. The jump is shocking.

[theArchLadder]

@stronnag So "sudo stronnag make me a video" does not require a password?! Cool! This jump happend from a 2 minutes poshold though, no wild movements... That makes it even more strange, is it usual for GPS units to have such crazy jumps on rare occasions?

[stronnag]

The video is uploading. I know you like flying fast, but 144m/s. Never seen that before.

[stronnag]

https://vimeo.com/166553671 Single stepping from around 1:50 is interesting ...

As is the fact that number of satellites and HDOP look good throughout. Reckon you'd have just hit the side of the building if you hadn't caught it.

[theArchLadder]

Awesome, thanks again! The home position looks about right, but as far as i can remember, i never flew east of my home position, so the position is still very wrong even after the big jump.

Another thought: AIR_1G = 9.8m/s², right? So it should take at least ~15 seconds to reach 144m/s, but it happend in an instant... I guess it doesn't work like that, so in what way is AIR_1G helpful compared to AIR_4G?

I also find it interesting that it only took inav 1.2 seconds or so to agree with the ~40 meter wrong gps position... Could we rely more on the inertial position estimator?

[digitalentity]

Ok, this looks like a serious GPS error - GPS position jumped about 10 meters in a split-second, but what's most disurbing that new position stayed biased. GPS glitch is usually a trasient event, but not in this case. I wonder what position is more accurate - before or after the glitch? Maybe you were unlucky to encounter an upload of new correction data to GPS satellites?

[digitalentity]

As far as I can tell INAV was behaving as expected. We need to think about more protection for these events in 1.2.

[theArchLadder]

@digitalentity Where did you get 10 meters from? Looks more like 40 meters to me. Position before the glitch looks very accurate to me. SBAS was set to NONE, so there should not have been any correction data AFAIK...?

I do agree that INAV did what it should with the data it had, but yeah, it would be nice to have some sort of protection against events like this.

[digitalentity]

@theArchLadder I mistyped, I was about to type 40, but missed the key and didn't read the comment before posting :smile:

I was referring to the event of correcting data onboard GPS satellites from ground control stations. As wikipedia tells us: "These updates synchronize the atomic clocks on board the satellites to within a few nanoseconds of each other, and adjust the ephemeris of each satellite's internal orbital model." I figure when that data is updated the GPS receiver on the ground will notice this as a jump in calculated position.

[theArchLadder]

This is what the gps graph looks like when flying like crazy from side to side in acro mode! The first graph is with PEDESTRIAN, the second graph is AIR_1G, AIR_4G looks equally nice, i guess i need to fly more agressive to ruin the AIR_1G graph!

PEDESTRIAN is not safe IMHO. But the glitch in my first post here looks very different, so i don't think it had anything to do with AIR_1G anymore...

Blackbox log, 1/5 is PEDESTRIAN, 2/5 is AIR_1G, 3/5 is AIR_4G. https://www.dropbox.com/s/4yog9m5wd8mht51/blackbox_log_2016-05-17_163714_pedestrian-limit-test?dl=0

[theArchLadder]

Made some position plots, red dots means low sat count. Unit on the numbers is meters. Both should look similar to the second one, but the first, PEDESTRIAN, is all over the place and the position is very wrong a few times, but there are no instant jumps in position

[digitalentity]

I believe this is no longer relevant - superceded by #512