High acceleration levels when running on Cube Orange (+)

[mwshafer]

Describe the bug

Reported acceleration levels are extreme when PX4 is run on a Cube Orange or Cube Orange +. Flight test results from both showing high acceleration can be found here.

I’ve ruled out hardware as the source of vibration through testing and by running Arducopter on my vehicle. That firmware doesn’t show these high acceleration levels in the log files. Log files for this back to back testing can be found here: Log files in google drive

This may only be an issue for the Cube Orange (+) controller, but I don’t know for sure.

To Reproduce

Load PX4 from main branch (or PX1.14/1.13) on a Cube Orange or Cube Orange+ and do a simple hover flight. Review flight logs.

Expected behavior

Acceptable acceleration values.

Screenshot / Media

Attached are two images of the accel levels for the back to back PX4 and Arducopter flight testing. According to the Ardupilot documentation, normal levels are below 15 m/s/s, which is what I am seeing in my log file.

arducopter_flight_accel.pdf

Flight Log

https://review.px4.io/plot_app?log=a9bebd55-10d4-42df-b12b-5781106c4696

Software Version

main

Flight controller

Pixcube Orange+

Vehicle type

Multicopter

How are the different components wired up (including port information)

No response

Additional context

No response

[julianoes]

Thanks for the report. We might have to check the filter settings of the IMUs. Does it fly ok though? And do you see any clipping in the log?

[mwshafer]

It is a bit twitchy in flight but hovers okay if the wind is low. I’ve done a bunch of flights and nearly crashed twice and crashed once. The crash was when I tried doing an autotune before I realized the root cause. The near crash today happened as it was descending into the prop wash during landing. You can see the bit oscillations at the end of the log file posed with this report.

As for clipping, how would I know? I’m not sure what I am looking for in the Flight Review plots for clipping. If clipping shows up as warnings in QGC, then no, I have not seen clipping.

[DronecodeBot]

This issue has been mentioned on Discussion Forum for PX4, Pixhawk, QGroundControl, MAVSDK, MAVLink. There might be relevant details there:

https://discuss.px4.io/t/high-vibration-in-cube-orange/32194/4

[AlexKlimaj]

Turn on high rate logging and see what the FFT shows. Then you can use the cutoffs and notch filters Then tune your pitch, it's oscillating.

[mwshafer]

How are we to know if that oscillation is from bad tuning of the pitch controller or from bad accel data causing the controller to oscillate in pitch? I don’t know how you run down root cause in this situation. I really don’t think it is a tuning issue because IMUs 1&2 should have levels below IMU 3 for the Cube Orange since they are isolated. The fact that have higher accel metrics suggests something deeper than tuning is causing this, right?

I’ve got some old flight logs where I had high rate logging turned on. I’ll post one tomorrow when I get into the office and can check my notes, as some were when I was flying my payload. Below is an example of one of them. I don’t see anything to be concerned about. Maybe I’m wrong.

[DronecodeBot]

This issue has been mentioned on Discussion Forum for PX4, Pixhawk, QGroundControl, MAVSDK, MAVLink. There might be relevant details there:

https://discuss.px4.io/t/high-vibration-in-cube-orange/32194/6

[sibujacob]

Hi, i too have a similar issue after updating the firmware from v.12.3 to v1.13.3 .... My graphs are in the following link

[mwshafer](https://github.com/mwshafer)

[julianoes]

Just noting that it's important not to focus too much on the absolute vibration amplitudes. While we generally want to keep the amplitudes low, high amplitudes by themselves are not a problem as such, as long as no clipping occurs.

Clipping can be found if you look at the log in plotjuggler:

There doesn't seem to be any (increasing) clipping here, except on landing which might be ok.

I would try reducing tuning if it's too twitchy, as well as adding a notch filter.

That being said, I will investigate the IMU filter settings and compare to other (non-Cube) flight controllers, to make sure that we aren't using completely different settings for Cube Orange than others.

[mwshafer]

@julianoes Thanks for looking into this. The Log Analysis using Flight Review | PX4 User Guide suggests that having Z values that overlap with the x and y values represents pretty extreme acceleration. Is this not the case?

The Orange and Orange+ have different IMUs. I am sure you are aware of this, but wanted to reiterate that fact since you will be looking at the IMU filter settings.

One final note, could part of the problem be that the Flight Review website still plots the vibration metrics red, yellow, and green levels with the values that were used in for PX4 <1.13 but since the metric amplitudes changed in 1.13, the website didn’t update to account for this?

Also, that clipping you are seeing happens before landing. In that flight, as the copter descended to land (in RTL mode) it entered into a series of very large and growing oscillations that I had to manually fly it out of. The clipping starts happening during those oscillations (it nearly flipped over).

[DronecodeBot]

This issue has been mentioned on Discussion Forum for PX4, Pixhawk, QGroundControl, MAVSDK, MAVLink. There might be relevant details there:

https://discuss.px4.io/t/high-vibration-in-cube-orange/32194/17

[julianoes]

The Log Analysis using Flight Review | PX4 User Guide suggests that having Z values that overlap with the x and y values represents pretty extreme acceleration. Is this not the case?

Right. You hear different things talking to different people. I'm not the expert on this. What I do know is that clipping is bad.

[julianoes]

The clipping starts happening during those oscillations (it nearly flipped over).

That's not good then. However, note that clipping usually can't be removed by software. If the range of a sensor is exceeded, that's a problem

[mwshafer]

Agreed. It is good that it isn’t clipping, but something is definitely wrong in the firmware because running Arducopter on the same drone doesn’t produces these huge raw acceleration values. My peak accel values running arducopter are never higher than 20m/s^2 but my PX4 values are peaking out at over 40 m/s^2. It almost seems like there is a scaling error in the received raw sensor data that makes the drone think it is accelerating harder than it really is.

[AlexKlimaj]

Its a filter and tuning issue I can tell you that with decent certainty. Once you have a high rate log I can take a look.

[mwshafer]

@AlexKlimaj I had switched over to Arducopter, so I needed to reinstall PX4. I just tried to install from QGC Daily and manually building it. Neither worked. Do you know if there is currently a problem with the 1.14 build? I posted my issues here.

[bresch]

IMUs 1&2 should have levels below IMU 3 for the Cube Orange since they are isolated

Note that the isolator in the Cube roughly acts as an underdamped 2nd order system, so if you feed noise at the resonant frequency, it will be amplified.

How are we to know if that oscillation is from bad tuning of the pitch controller or from bad accel data causing the controller to oscillate in pitch? I don’t know how you run down root cause in this situation.

It's hard to tell currently what it the cause of those high vibrations but there is visibly an oscillation on the pitch axis and reducing it might help to unclutter a bit the data.

or from bad accel data causing the controller to oscillate in pitch?

The angular rate controller is oscillating. It only uses the data from the gyro, not the accelerometer at all. And if the accel data would be causing attitude estimation issues, the oscillations would be much slower than this.

Also, sometimes the high frequency noise is simply amplified by the D-term of the angular rate PID controller and reducing a bit the D-term cutoff frequency might help (IMU_DGYRO_CUTOFF). We often reduce this on larger drones down to 12-20Hz.

[julianoes]

My conclusion is that the vibrations "look" big but as long as there is no accel clipping, this should be fine and any other issues would be from tuning and filter settings.

The Cube Orange and Orange+ uses some of the same sensors as other Pixhawks and also the same settings (mostly high rate and no on-device filtering). The filtering is then done on the PX4 side as far as I'm aware.

My test flights show "high accel vibrations" but no clipping and the flight behavior is good. I don't believe there is a difference between Cubes and other comparable Pixhawks, unless specific resonant frequencies of the internal vibration isolation was to be excited.

[skemp117]

Exact same issue here. Using cube orange + so we cannot downgrade to 1.12.x

[julianoes]

@skemp117 have you checked https://github.com/PX4/PX4-Autopilot/issues/21802#issuecomment-1630426720?

And have you tried latest v1.14.2?

[Jddl]

Same problem, any updates?

[julianoes]

@Jddl any logs that you can add? Which version?

[mwshafer]

Tying in the discussion from the Cube PX4 Forum here: https://discuss.px4.io/t/high-vibration-in-cube-orange/32194/81

I just did a series of test flights with a new drone and saw similar oscillations in pitch and roll (completely different airframe but of similar size). This was with V1.14.3. I tried dropping the MC_DGYRO_CUTOFF value to 12 and still saw the oscillations. I've also done a Autotune on the vehicle but the oscillations persist. @julianoes The logs from after the auto tuning are here: https://review.px4.io/plot_app?log=25005f93-d26c-4bd9-b556-76338395feee