AP 3.8rc5: CRUISE still gives heavy up/down oscillations (FBWA and RTL are ok)

marcmerlin commented 9 years ago

I've done about 8 flights with 3.4 and no problem so far, except in my last flight where for the first time, switching from FBWA to RTL caused some bad oscillations that were actually getting worse. After it looked so bad, I turned off RTL, went back to FBWA and it was stabilized again (and RTL worked ok later on in the flight).

Video worth a thousand words: https://youtu.be/UnuNdcZjwZ0?t=18m5s

Dataflash log: http://marc.merlins.org/tmp/MANU_to_GUIDED_4.BIN

gmorph commented 9 years ago

Thanks Marc - we will have a look.

tridge commented 9 years ago

Hi Marc, The most noticeable thing about the flight is the extremely noisy airspeed sensor data. It shows noise of around 4m/s peak-to-peak, which is about 10x the expected noise. My best guess is the sensor is poorly placed on the airframe. Certainly the resulting airspeed readings are quite suspect. Compounding the airspeed data problem was the target airspeed setting and the pitch limits.

the target airspeed was 12m/s, which I suspect is close to the lower limit of what the plane can fly at. Combine that with airspeed noise of 4m/s and it takes it over the edge.
the LIM_PITCH_MAX is 45 degrees, so it tries to put in a huge amount of elevator and pitch when switching to RTL if it wants to climb
at very low airspeed suddenly putting in a lot of elevator just stalls the horizontal stabiliser

If you want such a large LIM_PITCH_MAX for flying in FBWA then please use TECS_PITCH_MAX to limit the amount of up pitch in autonomous modes. I'd also suggest a significantly higher TRIM_ARSPD_CM to give you some more margin above the stall speed. Also look into your airspeed sensor install. Does it get prop wash? Is it too close to the edge of the wing? Is the cable close to a RF source like a FPV system? Cheers, Tridge

marcmerlin commented 9 years ago

First, thanks for looking at the data Tridge.

The most noticeable thing about the flight is the extremely noisy airspeed sensor data. It shows noise of around 4m/s peak-to-peak, which is about 10x the expected noise. My best guess is the sensor is poorly placed on the airframe. Certainly the resulting airspeed readings are quite suspect.

This is the airframe, front motor, airspeed sensor about 50cm from the plane body, short tube: http://marc.merlins.org/Pix/index.php?album=Flying/RC/20150815_BFG2600&img=551_BFG2600_Airspeed_MPX_Disconnect.jpg rest of the plane: http://marc.merlins.org/perso/rc/2015-08.html#BFG2600-FPV-build-with-Pixhawk-Ardupilot-3_3-_Diamond-2500-remake-from-Hobbyking_

Since the prop is a 13x7, I'm hoping the tube is sufficiently away from the prop disk washout. Do you think something might be wrong with my tube or 3DR airspeed sensor, or do you think it's simply still getting air from the prop?

Compounding the airspeed data problem was the target airspeed setting and the pitch limits.

the target airspeed was 12m/s, which I suspect is close to the lower limit of what the plane can fly at. Combine that with airspeed noise of 4m/s and it takes it over the edge.

12m/s = 43kph. That glider can fly level as slow as about 25kph, but not pitching up. But you're right that RTL brought the nose up quite a bit without much throttle so I guess it does look like it was stalling the wings.

the LIM_PITCH_MAX is 45 degrees, so it tries to put in a huge amount of elevator and pitch when switching to RTL if it wants to climb

True. Usuaully it adds quite a bit of power when it does this, but it didn't in this case, hence the problem.

at very low airspeed suddenly putting in a lot of elevator just stalls the horizontal stabiliser If you want such a large LIM_PITCH_MAX for flying in FBWA then please use TECS_PITCH_MAX to limit the amount of up pitch in autonomous modes. I'd also suggest a significantly higher TRIM_ARSPD_CM to give you some more margin above the stall speed.

Those are good suggestions, I will do this. But do you know why AP used to be able to go way up and it knew to put a boatload of power to pitch up that much, and in this case it failed to do so? https://www.youtube.com/watch?v=UnuNdcZjwZ0&feature=youtu.be&t=18m5s clearly show that RTL barely put any throttle while pitching up, somehow it ought to know it needs more than just a few percent power, should it not? I also notice from the video that it's quite slow at adding power and while I did cut it after 4 oscillations, I can see that after stalling, it just tries even more nose up without much power at all. I have enough power on this that I can alsmot fly 90 degrees up at full power.

I can't say if it's a bug or not, so I'll leave that up to you to decide. All I can say is that with AP 3.3 I never witnessed this and I had the same LIM_PITCH_MAX from what I remember.

Then this might also explain why when I tried stall avoidance in TRAINING mode, I was able to pull the nose up a lot and get the plane to stall without problems (or maybe this is 100% unrelated, not sure).

Also look into your airspeed sensor install. Does it get prop wash? Is it too close to the edge of the wing? Is the cable close to a RF source like a FPV system?

The I2C wire does go close to other wires, but it's digital, so RF shouldn't affect it, should it?

Either way, the radios are: 1) 433 LRS, not too far from pixhawk 2) 5Ghz VTX, all the way on the tail 3) 3DR 433Mhz telemetry is unplugged since it conflicts with LRS Picture: http://marc.merlins.org/blogimg/thumb1024_111_BFG2600_Airframe.jpg

Let me know if I can provide other info, and thanks again for reviewing this.

Marc

"A mouse is a device used to point at the xterm you want to type in" - A.S.R. Microsoft is to operating systems .... .... what McDonalds is to gourmet cooking Home page: http://marc.merlins.org/

marcmerlin commented 9 years ago

Actually I was just able to retrieve some screenshots from the OSD (recording is incomplete, but those are enough to help).

First shot, I'm happily flying in FBWA, 2 important things to note: 1) I'm above the target RTL altitude of 100m 2) I'm directly flying towards the RTL location So RTL does not need to turn, and does not need to climb. shot0008

Yet, it tries both and fails. a) isn't it a bug that RTL tried to climb when it did not need to? (unless it believes that terrain ahead is not flat due to the mountains on the right) b) it applies 36% throttle which is 6A out of 45A possible (it's not linear I guess). 36% is definitely not enough but it doesn't seem to know that (maybe a misconfiguration, although I'd assume it can also auto learn that) shot0010

23 degrees pit up, only 33% throttle, 2.75A only, this means the prop creates almost no thrust. Also note again how HA is 101M is I'm still above my target RTL altitude. RTL does not need to climb. shot0011

27 degrees pitch up, throttle 0%. I'm not even sure what to say shot0012

Also minimOSD shows airspeeds of 52, 58, and 55kph in the 3 RTL shots. I do believe those airspeeds have to be wrong since I was doing 42kph pitching down, and those are pitch up with very little throttle. This could indicate that the prop, even when it's not making much thrust, could be foiling the airspeed sensor. However, my takeoffs seem to negate that assumption. See 100% throttle, 46A, AS only 39kph with a pitch up of 27 degrees. Although either the GS, AS, or wind speed are wrong since they don't add up. shot0002

magicrub commented 9 years ago

The GS, AS and wind speed won't add up unless the wind vector is exactly in line with flight. What is shown on the screen is the wind x, y vector length.

Do you fly in auto much? RTL is pretty much auto mode so I imagine any problems you potentially have, if any, that are Airframe/pitot works happen during an auto flight too.

Since you are above the RTL location you should five down. However since it thinks you are going too fast it may be pitching up to bleed off some speed.

Do you have stall prevention enabled? What is your stall threshold m/s? I'll try and take a look at the log if I have some time tomorrow. On Oct 18, 2015 11:20 PM, "Marc MERLIN" notifications@github.com wrote:

Actually I was just able to retrieve some screenshots from the OSD (recording is incomplete, but those are enough to help).

First shot, I'm happily flying in FBWA, 2 important things to note: 1) I'm above the target RTL altitude of 100m 2) I'm directly flying towards the RTL location So RTL does not need to turn, and does not need to climb. [image: shot0008] https://cloud.githubusercontent.com/assets/1369412/10570846/4b7e0230-75ed-11e5-8c18-91c5db0e543b.png

Yet, it tries both and fails. a) It's a bug that RTL tried to climb when it did not need to b) it applies 36% throttle which is 6A out of 45A possible (it's not linear I guess). 36% is definitely not enough but it doesn't seem to know that (maybe a misconfiguration, although I'd assume it can also auto learn that) [image: shot0010] https://cloud.githubusercontent.com/assets/1369412/10570843/4b7bb5d4-75ed-11e5-91ee-fdb4717c3abd.png

23 degrees pit up, only 33% throttle, 2.75A only, this means the prop creates almost no thrust. Also note again how HA is 101M is I'm still above my target RTL altitude. RTL does not need to climb. [image: shot0011] https://cloud.githubusercontent.com/assets/1369412/10570845/4b7dba78-75ed-11e5-825c-a86f4c0f1b27.png

27 degrees pitch up, throttle 0%. I'm not even sure what to say [image: shot0012] https://cloud.githubusercontent.com/assets/1369412/10570844/4b7d54ac-75ed-11e5-8f08-b015abb27454.png

Also minimOSD shows airspeeds of 52, 58, and 55kph in the 3 RTL shots. I do believe those airspeeds have to be wrong since I was doing 42kph pitching down, and those are pitch up with very little throttle. This could indicate that the prop, even when it's not making much thrust, could be foiling the airspeed sensor. However, my takeoffs seem to negate that assumption. See 100% throttle, 46A, AS only 39kph with a pitch up of 27 degrees. Although either the GS, AS, or wind speed are wrong since they don't add up. [image: shot0002] https://cloud.githubusercontent.com/assets/1369412/10570967/d26ed5fc-75ee-11e5-9270-aa3738b3631d.png

— Reply to this email directly or view it on GitHub https://github.com/diydrones/ardupilot/issues/3032#issuecomment-149113528 .

marcmerlin commented 9 years ago

@magicrub you're correct about AS GS and wind, but the last snapshot shows a tailwind pushing me and GS being slower than AS which obviously isn't possible. However looking at the wind later during the flight it looks like the wind indication was just wrong after takeoff (but since it's computed from GS and AS, it ought to be right almost right away if you have an airspeed indicator)

I never fly AUTO missions. I do have stall prevention enabled, but it's supposed to be active both in FBWA an RTL, so I'm not sure it should come into effect differently between the two modes. ARSPD_FBW_MIN = 12 (43kph) and MAX = 22 (80kph). I was nowhere close to that speed of course. I had ARSPD_AUTOCAL still set to 1, so I just set it back to 0, but AP shouldn't have been thinking I was going too fast since I was way below FBW_MAX (and besides this speed limit would also have impacted FBW which I was in). Note that I put my stall speed way above my real stall, but this was to test stall code in TRAINING mode.

Thanks for your reply and having a look.

marcmerlin commented 9 years ago

While the dataflash above should hopefully have enough info (along with the screenshots) to show that things didn't work like they should have (ardupilot pitches up even though it doesn't need to get altitude, and then pitches up a lot without applying enough motor power, actually causing stalls), is there anything else I can provide? From what I can tell, there are 2 different bugs here, even if we're not clear about the airspeed sensor being fully correct, it was showing enough airspeed.

marcmerlin commented 8 years ago

@magicrub while this isn't a crash bug yet, I feel like it's only a matter of time before it becomes one.

While usually things just work, today I had 2 flights where AP 3.4 misbehaved badly again. It's bad enough that if I didn't have video and manual control, the plane would never have flown back and likely crashed either due to low battery or unrecoverable stall.

ARSPD_FBW_MIN is 12m/s. Maybe it's not enough, so I'll raise it to 15, but somehow I feel like there are other bugs that are causing uncontrolled nose up without power, and creating those stalls.

Example 1: CRUISE failure: https://youtu.be/y-TKmxJ5Tyg?t=10s I'm not sure why CRUISE was commanding such a pitch up attitude causing repeated stalls, and with only 22% throttle. Usually it puts a lot of power when it goes up, and here virtually none (I see 1.93A draw, that means the motor is barely turning). So why 1) so much pitch up 2) no motor power 3) no detection of the stalls and attempt to regain

Example 2: RTL failure https://youtu.be/uD5Ou2tTKjM?t=1m41s Pausing on a non snowy frame shows pitch up of 23 degrees, no idea why, and RTL missing the target heading by more than 90 degrees. I can see how eventually the airspeed drops enough to do this, but there is no reason for RTL to fly so erratically with such a nose up altitude and not full power. And if it gets into that mode, shouldn't it detect the stalls, and add power? Switch to CRUISE at 2:19 fixes the problem

marcmerlin commented 8 years ago

http://marc.merlins.org/tmp/Cruise_Stall_4.BIN http://marc.merlins.org/tmp/RTL_Stall_3.BIN

marcmerlin commented 8 years ago

Hi @tridge as discussed right now at LCA, this is the bug that caused me to raise ARSPD_FBW_MIN to 14 when i had TRIM_ARSPD_CM which I forgot was still at 12. My airframe starts stalling around 10m/s, but due to the behaviour I got in this ssue, I raised it to 14 to see if that helped.