AP 3.8rc5: CRUISE still gives heavy up/down oscillations (FBWA and RTL are ok)

[marcmerlin]

I've done about 8 flights with 3.4 and no problem so far, except in my last flight where for the first time, switching from FBWA to RTL caused some bad oscillations that were actually getting worse. After it looked so bad, I turned off RTL, went back to FBWA and it was stabilized again (and RTL worked ok later on in the flight).

Video worth a thousand words: https://youtu.be/UnuNdcZjwZ0?t=18m5s

Dataflash log: http://marc.merlins.org/tmp/MANU_to_GUIDED_4.BIN

[gmorph]

Thanks Marc - we will have a look.

[tridge]

Hi Marc, The most noticeable thing about the flight is the extremely noisy airspeed sensor data. It shows noise of around 4m/s peak-to-peak, which is about 10x the expected noise. My best guess is the sensor is poorly placed on the airframe. Certainly the resulting airspeed readings are quite suspect. Compounding the airspeed data problem was the target airspeed setting and the pitch limits.

• the target airspeed was 12m/s, which I suspect is close to the lower limit of what the plane can fly at. Combine that with airspeed noise of 4m/s and it takes it over the edge.
• the LIM_PITCH_MAX is 45 degrees, so it tries to put in a huge amount of elevator and pitch when switching to RTL if it wants to climb
• at very low airspeed suddenly putting in a lot of elevator just stalls the horizontal stabiliser

If you want such a large LIM_PITCH_MAX for flying in FBWA then please use TECS_PITCH_MAX to limit the amount of up pitch in autonomous modes. I'd also suggest a significantly higher TRIM_ARSPD_CM to give you some more margin above the stall speed. Also look into your airspeed sensor install. Does it get prop wash? Is it too close to the edge of the wing? Is the cable close to a RF source like a FPV system? Cheers, Tridge

[marcmerlin]

First, thanks for looking at the data Tridge.

The most noticeable thing about the flight is the extremely noisy airspeed sensor data. It shows noise of around 4m/s peak-to-peak, which is about 10x the expected noise. My best guess is the sensor is poorly placed on the airframe. Certainly the resulting airspeed readings are quite suspect.

This is the airframe, front motor, airspeed sensor about 50cm from the plane body, short tube: http://marc.merlins.org/Pix/index.php?album=Flying/RC/20150815_BFG2600&img=551_BFG2600_Airspeed_MPX_Disconnect.jpg rest of the plane: http://marc.merlins.org/perso/rc/2015-08.html#BFG2600-FPV-build-with-Pixhawk-Ardupilot-3_3-_Diamond-2500-remake-from-Hobbyking_

Since the prop is a 13x7, I'm hoping the tube is sufficiently away from the prop disk washout. Do you think something might be wrong with my tube or 3DR airspeed sensor, or do you think it's simply still getting air from the prop?

Compounding the airspeed data problem was the target airspeed setting and the pitch limits.

• the target airspeed was 12m/s, which I suspect is close to the lower limit of what the plane can fly at. Combine that with airspeed noise of 4m/s and it takes it over the edge.

12m/s = 43kph. That glider can fly level as slow as about 25kph, but not pitching up. But you're right that RTL brought the nose up quite a bit without much throttle so I guess it does look like it was stalling the wings.

• the LIM_PITCH_MAX is 45 degrees, so it tries to put in a huge amount of elevator and pitch when switching to RTL if it wants to climb

True. Usuaully it adds quite a bit of power when it does this, but it didn't in this case, hence the problem.

• at very low airspeed suddenly putting in a lot of elevator just stalls the horizontal stabiliser If you want such a large LIM_PITCH_MAX for flying in FBWA then please use TECS_PITCH_MAX to limit the amount of up pitch in autonomous modes. I'd also suggest a significantly higher TRIM_ARSPD_CM to give you some more margin above the stall speed.

Those are good suggestions, I will do this. But do you know why AP used to be able to go way up and it knew to put a boatload of power to pitch up that much, and in this case it failed to do so? https://www.youtube.com/watch?v=UnuNdcZjwZ0&feature=youtu.be&t=18m5s clearly show that RTL barely put any throttle while pitching up, somehow it ought to know it needs more than just a few percent power, should it not? I also notice from the video that it's quite slow at adding power and while I did cut it after 4 oscillations, I can see that after stalling, it just tries even more nose up without much power at all. I have enough power on this that I can alsmot fly 90 degrees up at full power.

I can't say if it's a bug or not, so I'll leave that up to you to decide. All I can say is that with AP 3.3 I never witnessed this and I had the same LIM_PITCH_MAX from what I remember.

Then this might also explain why when I tried stall avoidance in TRAINING mode, I was able to pull the nose up a lot and get the plane to stall without problems (or maybe this is 100% unrelated, not sure).

Also look into your airspeed sensor install. Does it get prop wash? Is it too close to the edge of the wing? Is the cable close to a RF source like a FPV system?

The I2C wire does go close to other wires, but it's digital, so RF shouldn't affect it, should it?

Either way, the radios are: 1) 433 LRS, not too far from pixhawk 2) 5Ghz VTX, all the way on the tail 3) 3DR 433Mhz telemetry is unplugged since it conflicts with LRS Picture: http://marc.merlins.org/blogimg/thumb1024_111_BFG2600_Airframe.jpg

Let me know if I can provide other info, and thanks again for reviewing this.

Marc

"A mouse is a device used to point at the xterm you want to type in" - A.S.R. Microsoft is to operating systems .... .... what McDonalds is to gourmet cooking Home page: http://marc.merlins.org/

[marcmerlin]

Actually I was just able to retrieve some screenshots from the OSD (recording is incomplete, but those are enough to help).

First shot, I'm happily flying in FBWA, 2 important things to note: 1) I'm above the target RTL altitude of 100m 2) I'm directly flying towards the RTL location So RTL does not need to turn, and does not need to climb.

Yet, it tries both and fails. a) isn't it a bug that RTL tried to climb when it did not need to? (unless it believes that terrain ahead is not flat due to the mountains on the right) b) it applies 36% throttle which is 6A out of 45A possible (it's not linear I guess). 36% is definitely not enough but it doesn't seem to know that (maybe a misconfiguration, although I'd assume it can also auto learn that)

23 degrees pit up, only 33% throttle, 2.75A only, this means the prop creates almost no thrust. Also note again how HA is 101M is I'm still above my target RTL altitude. RTL does not need to climb.

27 degrees pitch up, throttle 0%. I'm not even sure what to say

Also minimOSD shows airspeeds of 52, 58, and 55kph in the 3 RTL shots. I do believe those airspeeds have to be wrong since I was doing 42kph pitching down, and those are pitch up with very little throttle. This could indicate that the prop, even when it's not making much thrust, could be foiling the airspeed sensor. However, my takeoffs seem to negate that assumption. See 100% throttle, 46A, AS only 39kph with a pitch up of 27 degrees. Although either the GS, AS, or wind speed are wrong since they don't add up.

[magicrub]

The GS, AS and wind speed won't add up unless the wind vector is exactly in line with flight. What is shown on the screen is the wind x, y vector length.

Do you fly in auto much? RTL is pretty much auto mode so I imagine any problems you potentially have, if any, that are Airframe/pitot works happen during an auto flight too.

Since you are above the RTL location you should five down. However since it thinks you are going too fast it may be pitching up to bleed off some speed.

Do you have stall prevention enabled? What is your stall threshold m/s? I'll try and take a look at the log if I have some time tomorrow. On Oct 18, 2015 11:20 PM, "Marc MERLIN" notifications@github.com wrote:

Actually I was just able to retrieve some screenshots from the OSD (recording is incomplete, but those are enough to help).

First shot, I'm happily flying in FBWA, 2 important things to note: 1) I'm above the target RTL altitude of 100m 2) I'm directly flying towards the RTL location So RTL does not need to turn, and does not need to climb. [image: shot0008] https://cloud.githubusercontent.com/assets/1369412/10570846/4b7e0230-75ed-11e5-8c18-91c5db0e543b.png

Yet, it tries both and fails. a) It's a bug that RTL tried to climb when it did not need to b) it applies 36% throttle which is 6A out of 45A possible (it's not linear I guess). 36% is definitely not enough but it doesn't seem to know that (maybe a misconfiguration, although I'd assume it can also auto learn that) [image: shot0010] https://cloud.githubusercontent.com/assets/1369412/10570843/4b7bb5d4-75ed-11e5-91ee-fdb4717c3abd.png

23 degrees pit up, only 33% throttle, 2.75A only, this means the prop creates almost no thrust. Also note again how HA is 101M is I'm still above my target RTL altitude. RTL does not need to climb. [image: shot0011] https://cloud.githubusercontent.com/assets/1369412/10570845/4b7dba78-75ed-11e5-825c-a86f4c0f1b27.png

27 degrees pitch up, throttle 0%. I'm not even sure what to say [image: shot0012] https://cloud.githubusercontent.com/assets/1369412/10570844/4b7d54ac-75ed-11e5-8f08-b015abb27454.png

Also minimOSD shows airspeeds of 52, 58, and 55kph in the 3 RTL shots. I do believe those airspeeds have to be wrong since I was doing 42kph pitching down, and those are pitch up with very little throttle. This could indicate that the prop, even when it's not making much thrust, could be foiling the airspeed sensor. However, my takeoffs seem to negate that assumption. See 100% throttle, 46A, AS only 39kph with a pitch up of 27 degrees. Although either the GS, AS, or wind speed are wrong since they don't add up. [image: shot0002] https://cloud.githubusercontent.com/assets/1369412/10570967/d26ed5fc-75ee-11e5-9270-aa3738b3631d.png

— Reply to this email directly or view it on GitHub https://github.com/diydrones/ardupilot/issues/3032#issuecomment-149113528 .

[marcmerlin]

@magicrub you're correct about AS GS and wind, but the last snapshot shows a tailwind pushing me and GS being slower than AS which obviously isn't possible. However looking at the wind later during the flight it looks like the wind indication was just wrong after takeoff (but since it's computed from GS and AS, it ought to be right almost right away if you have an airspeed indicator)

I never fly AUTO missions. I do have stall prevention enabled, but it's supposed to be active both in FBWA an RTL, so I'm not sure it should come into effect differently between the two modes. ARSPD_FBW_MIN = 12 (43kph) and MAX = 22 (80kph). I was nowhere close to that speed of course. I had ARSPD_AUTOCAL still set to 1, so I just set it back to 0, but AP shouldn't have been thinking I was going too fast since I was way below FBW_MAX (and besides this speed limit would also have impacted FBW which I was in). Note that I put my stall speed way above my real stall, but this was to test stall code in TRAINING mode.

Thanks for your reply and having a look.

[marcmerlin]

While the dataflash above should hopefully have enough info (along with the screenshots) to show that things didn't work like they should have (ardupilot pitches up even though it doesn't need to get altitude, and then pitches up a lot without applying enough motor power, actually causing stalls), is there anything else I can provide? From what I can tell, there are 2 different bugs here, even if we're not clear about the airspeed sensor being fully correct, it was showing enough airspeed.

[marcmerlin]

@magicrub while this isn't a crash bug yet, I feel like it's only a matter of time before it becomes one.

While usually things just work, today I had 2 flights where AP 3.4 misbehaved badly again. It's bad enough that if I didn't have video and manual control, the plane would never have flown back and likely crashed either due to low battery or unrecoverable stall.

ARSPD_FBW_MIN is 12m/s. Maybe it's not enough, so I'll raise it to 15, but somehow I feel like there are other bugs that are causing uncontrolled nose up without power, and creating those stalls.

Example 1: CRUISE failure: https://youtu.be/y-TKmxJ5Tyg?t=10s I'm not sure why CRUISE was commanding such a pitch up attitude causing repeated stalls, and with only 22% throttle. Usually it puts a lot of power when it goes up, and here virtually none (I see 1.93A draw, that means the motor is barely turning). So why 1) so much pitch up 2) no motor power 3) no detection of the stalls and attempt to regain

Example 2: RTL failure https://youtu.be/uD5Ou2tTKjM?t=1m41s Pausing on a non snowy frame shows pitch up of 23 degrees, no idea why, and RTL missing the target heading by more than 90 degrees. I can see how eventually the airspeed drops enough to do this, but there is no reason for RTL to fly so erratically with such a nose up altitude and not full power. And if it gets into that mode, shouldn't it detect the stalls, and add power? Switch to CRUISE at 2:19 fixes the problem

[marcmerlin]

http://marc.merlins.org/tmp/Cruise_Stall_4.BIN http://marc.merlins.org/tmp/RTL_Stall_3.BIN

[marcmerlin]

Hi @tridge as discussed right now at LCA, this is the bug that caused me to raise ARSPD_FBW_MIN to 14 when i had TRIM_ARSPD_CM which I forgot was still at 12. My airframe starts stalling around 10m/s, but due to the behaviour I got in this ssue, I raised it to 14 to see if that helped.

[OXINARF]

Is this still relevant?

[marcmerlin]

I still get oscillations in CRUISE on 2 of my aircraft, but not to a point that it will stall, just enough to be annoying, not enough to be a risk. However, this could be a PID issue with the elevator control, although I don't have the problem in auto takeoff or RTL, just CRUISE. Last test was with 3.7 though since I just got 3.8rc5 working on my aircraft yesterday (on the bench that is)

[OXINARF]

@marcmerlin Can you update the title please?

[marcmerlin]

How about this. I'll do test this carefully on my next flights, give more details on what I saw and then update title to match what I've seen. Right now I'm not 100% certain if RTL is affected or not.

[OXINARF]

OK, thanks. I've re-added the NeedAnswer label just to make sure we keep track of it.

[marcmerlin]

On one of my aircrafts, 3.8b5 CRUISE worked well enough. On the other one, it was terrible (BFG2600) https://youtu.be/n4iWlvrRA5E?t=11m you can check the time period 14:30:50 to 14:32:22 using the timestamps in mavexplorer or look for the CRUISE 89s after FBWA 113s Not only does the aircraft pitch up and down pretty violently, but the motor oscillates between idle and close to full power. Now, maybe the aircraft is slightly out of (auto)tune, but I did tune it more than once, in the past, including after this autotune induced flip over: https://github.com/ArduPilot/ardupilot/issues/4288

FBWA worked beautifully as always, although the heading drifted because it doesn't keep that. I actually really wish I had a pilot throttle controlled CRUISE mode, where the heading would lock, but I'd stay in control of pitch and motor power (great for a motorglider)

[marcmerlin]

http://marc.merlins.org/tmp/00000033_BFG_CRUISE_OSCILLATION.BIN

[marcmerlin]

@OXINARF did the new logs I update help? Is there anything you'd like me to try before my next flight?

[marcmerlin]

@magicrub just in case you're interested in this issue too with the 3.8 release not that far away. I'm ok doing another autotune on my next flight and hopefully it won't fail badly this time, but somehow it seems to be more than just an autotune issue (although let me know if you think I'm wrong on this), especially the part where it kept spinning the motor up and down so much, being constantly behind the aircraft and reacting too late. I also forgot to mention that auto takeoffs work beautifully, it keeps the right pitch up and just goes, just like RTL seems to behave ok (and those two modes of course also control the motor and elevator)

[marcmerlin]

@WickedShell my apologies, I meant @magicrub

[OXINARF]

@marcmerlin I should clarify that my knowledge of Plane is next to 0, so I can't help reviewing the log. But this issue, together with the other ones you have revisited, are on my list for Tridge to look at.

[marcmerlin]

Ok, as an update, I re calibrated my BFG2600 aircraft again, and cruise was significantly better. Maybe it got out of tune somehow, hard to say (and kind of unexpected if so).

I'm attaching the tuning changes that autotune did. Doesn't look like nothing major, but seems to have helped:

I'll let you decide if you feel that this was the only problem or if the small mistune should not have been enough of a reason for the pretty violent up/down and motor off/on that were happening.

[peterbarker]

We've flown cruise a lot since this issue was created and haven't seen the behaviour reported here. Looks like it might have come down to that tuning. We also have a new autotune system in beta...