search

pidtuner / pidtuner.github.io

Free PID Controller gains tuning tool
GNU General Public License v3.0
239 stars 27 forks source link

support Cleanflight/Betaflight/INAV logs #12

Open naorunaoru opened 3 years ago

naorunaoru commented 3 years ago

Betaflight is currently one of the most popular software solutions for racing/freestyle quadcopters. As far as I know, all efforts to make automatic PID tuning weren't sufficiently successful so none of the Cleanflight forks pursue this goal now.

I couldn't find exact description of the file format, but here are some links:

Unfortunately I don't have any sample blackbox logs on hand so I'll try to grab some later this week.

Please ask if you have any questions.

pidtuner commented 3 years ago

I have looked really fast, and it seems there is indeed a lot of interest for PID tuning for quads. Having said that, I have (physically) no hardware to start toying around. So we can start with this:

After taking actions on the points above I could make a video tutorial series on how to tune PIDs for quads using the pidtuner tool.

mituritsyn commented 3 years ago

Hi,

How could I, given the hardware/software and after making some experiments, retrieve the data that comes in and out of the PIDs

You need flight controller with blackbox, here is log file example: https://drive.google.com/file/d/1b6vZMn3iw7ZSHXl1-pAcHbg8-kqShHCz/view?usp=sharing You can view it through blackbox log viewer: https://github.com/betaflight/blackbox-log-viewer or pid toolbox: https://github.com/bw1129/PIDtoolbox/releases Also there is a tool to decode bbl files to csv format: https://github.com/bw1129/PIDtoolbox/releases

Help me find out information about the overall control problem

there is PID + FF system with some improvements such as I-term relax, D-min(D-boost), PI-boost called antigravity, Absolute Control - calculating I-term constatn component. And It's not trivial to tune them all)

Recommend some beginner hardware

unfortunately besides a quad you'll need a bunch of pretty expensive equipment such as radio TX batteries chargers etc all round will cost you at least 300-500$

Help me find in software, exactly where are the PIDs implemented you could start from pid.h pid.c files here: https://github.com/betaflight/betaflight/tree/master/src/main/flight

pidtuner commented 3 years ago

My current patreon income is 25$ per month, so maybe in a year or two I could afford the hardware :upside_down_face:

In the meanwhile, the simplest, although not the best, is to try with existing closed loop data.

Let's just focus in one PID.

If any of you guys have test data where you drove a quad aggressively and have a log, we can do the following:

  1. Convert the log to CSV.
  2. Delete all columns except (time, pid input, pid output, pid reference).
  3. Post the resulting CSV here, along the value of the PID gains used in the experiment.

With this data we can start some discussion without the need of hardware.

naorunaoru commented 3 years ago

25$ per month

Not anymore. :)

pidtuner commented 3 years ago

You need flight controller with blackbox, here is log file example: https://drive.google.com/file/d/1b6vZMn3iw7ZSHXl1-pAcHbg8-kqShHCz/view?usp=sharing You can view it through blackbox log viewer: https://github.com/betaflight/blackbox-log-viewer or pid toolbox:

@mituritsyn I have poked around your log file example, converted to CSV, and tried to match the column names to the betaflight source code but have hard time matching them.

What we need for PID tuning, from the source code point of view, is the gyroRate variable, the pidData[axis].Sum variable and the time. Since we have closed loop data, it would be also useful to have the currentPidSetpoint.

Form what I have seen, there are PIDs for FD_ROLL, FD_PITCH and FD_YAW axis, but have not been able to match to the log columns:

H Field I name:loopIteration,time,axisP[0],axisP[1],axisP[2],axisI[0],axisI[1],axisI[2],axisD[0],axisD[1],axisF[0],axisF[1],axisF[2],rcCommand[0],rcCommand[1],rcCommand[2],rcCommand[3],setpoint[0],setpoint[1],setpoint[2],setpoint[3],gyroADC[0],gyroADC[1],gyroADC[2],debug[0],debug[1],debug[2],debug[3],motor[0],motor[1],motor[2],motor[3]

I traced most of the PID source code to the @etracer65 github user, maybe he can help us out.

mituritsyn commented 3 years ago

axisP[0],axisP[1],axisP[2],axisI[0],axisI[1],axisI[2],axisD[0],axisD[1],axisF[0],axisF[1],axisF[2]

here is roll pitch yaw PIDs

setpoint[0],setpoint[1],setpoint[2],setpoint[3]

roll pitch yaw throttle setpoints

Not sure about gyroADS[ ] values. Most likely they are filtered gyro rates. what am I sure about is that debug[0-3] is gyro prefiltered values for roll pitch and yaw accordingly

mituritsyn commented 3 years ago

here is what I found in dosc folder: gyroADC/8192*2000 = deg/s

pidtuner commented 3 years ago

Browsing the code a little further found out where the variables are logged, it is a shame since it seems the data is truncated to integers before being logged. This is not ideal for PID tuning.

It also seems that pidData[axis].Sum is not being logged, but their individual P, I and D contributions are, so that is fine because we can reconstruct the PID output.

In the meanwhile, we need blackbox logs examples where data was recorded, ideally indoor or with no wind or external disturbances. Preferably with badly tuned PID gains, which give better information. Also trying to shake the drone during the experiment, move it around to get more dynamic information.

I found this simulator that appears to be betaflight compatible:

Anybody recommends it? Has a license?

mituritsyn commented 3 years ago

the data is truncated to integers before being logged

probably this is needed to further data compression. In BF pids are working at 1-8kHz rate and even partial logging requre huge bandwith. We tune pids via blackbox log viever analisys and it's pretty enought.

Also trying to shake the drone during the experiment

pretty bad idea to shake racing drone during flight)

https://www.team-blacksheep.com/simulator

it's best sim for drone racers but as i noticed it hasn't good enought gyro noize and pid simulation so it doesn't fit for pid tune simulation. here is one BF compatible sim: https://store.steampowered.com/app/410340/Liftoff_FPV_Drone_Racing/https://store.steampowered.com/app/410340/Liftoff_FPV_Drone_Racing/

we need blackbox logs examples

it's a rainy week so I cant provide more data

pidtuner commented 3 years ago

Sorry, I did not make myself clear, what I mean with "shake" is not with the hand or anything physically, I mean moving the drone around with the RC. And ideally indoors, to avoid external perturbations (link wind).

I been looking around how people do it and some have the right idea, like these guys:

What we are looking for as indeed the closest to a "step response" data.

@mituritsyn, @naorunaoru, you think one of these could work? They list Betaflight as firmware, what I am not sure is if I can save blackbox logs with it or if I need to buy something extra.

pidtuner commented 3 years ago

@bw1129, I saw from your video you are an expert in quad PID tuning, care to contribute? :slightly_smiling_face:

bw1129 commented 3 years ago

Very nice app!

naorunaoru commented 3 years ago

you think one of these could work?

Blackbox is a feature of the flight controller. Taycans use FCs without blackbox unfortunately, I guess due to high availability of PID presets for this class of drones.

If you want something similar, there are iFlight Protek 25 and 35, both have FCs with 8 MB of onboard flash which should be enough for a few flights.

mituritsyn commented 3 years ago

you think one of these could work?

These ones are specialized for cinematic filming. They fly with very low turn rates and doesn't needed in sharp tuned pids at all. I'd prefer something more agressive) for example: https://banggood.com/Diatone-Roma-F5-Lite-4S-5-Inch-Freestyle-FPV-Racing-Drone-PNP-CADDX-RATEL-Cam-MAMBA-F405-DJI-Lite-FC-400MW-VTX-40A-ESC-2450KV-Motor-p-1740586.html

mituritsyn commented 3 years ago

What we are looking for as indeed the closest to a "step response" data.

here is some old logs with steps for tuning: https://drive.google.com/drive/folders/1BkVSeszN4sz0ZQsZ-SJZ0V0VDp2F4yCp?usp=sharing @spatzengr I'm sure you can provide more data)

pidtuner commented 3 years ago

@mituritsyn, @naorunaoru, thanks for the recommendations.

The iFlight Protek seems better for me because I plan to do indoor tests to do research. First because, as in @bw1129 video, identification tests are cleaner when there are no external disturbances (no wind) and all non-linearities are minimized (Feedforward and other non-standard PID featured disabled).

I saw from the banggood listings, that they sell the drone + receiver combo, I suppose this means I must also buy a matching transmitter, could you please help me choose the cheapest transmitter-receiver combo that will do the job? Note I am not looking for the best hardware, most performance, just the cheapest that can do the job (Betaflight PID configuration and Blackbox).

The idea would be to provide a step-by-step procedure for PID tuning roughly as follows:

  1. How to configure Betaflight parameters to prepare for PID tuning test.
  2. How to perform indoors tuning test.
  3. How to pre-process blackbox log file to prepare for PID tuner tool.
  4. How to use PID Tuner tool to obtain PID gains.
  5. How to convert standard PID gains to Betaflight PID gains (Betaflight does not use standard gains, it performs scaling and other transformations).
  6. How to validate good tuning.

If I succeed in defining these steps in a cheap drone, they should generalize to any other drone using Betaflight.

Steps 1,2 and 6 are pretty much already described in @bw1129 video. His tool is very good in showing an analyzing closed loop step response (setpoint to gyro, which is the result of PID tuning). What we need for PID tuning (get good gains in one step) is obtaining an open loop response (PID output to gyro).

Thanks for your help, @mituritsyn I am going to take a look at your logs as soon as I can.

naorunaoru commented 3 years ago

could you please help me choose the cheapest transmitter-receiver combo that will do the job?

Radiomaster TX12 is hard to beat for the price. If you want to go lower, there's Jumper T-Lite. Even lower — BetaFPV LiteRadio, but be aware that it comes with either Frsky or Bayang compatibility and you don't want the Bayang. Also it's a bit cumbersome to set up because there's no screen.

Other options may include Radiomaster TX16 or a comparable offer from Radioking, the first will serve you well, the second may be just enough to get by.

If you are opting for a remote with reduced multiprotocol module (often marked as CC2500), you can pair it basically only with Frsky D8 and D16 compatible receivers. Sure, Futaba is on the list too, but their receivers are prohibitively expensive.

pidtuner commented 3 years ago

OK, so to confirm, I can get started with these two?

image

image

pidtuner commented 3 years ago

Hi @mituritsyn, thanks for the logs, I only managed to get some slightly useful information for the Yaw loop, because it is the only one that had D gain disabled during the test.

I am sure if we make a Basement Tuning blackbox log with Betaflight settings as in @bw1129 's video, we could get a much better fit, which would yield optimal PID gains.

mituritsyn commented 3 years ago

FrSky R-XSR or Frsky XM+

xm+ should be more than enought for indoor flight

Yaw loop, because it is the only one that had D gain disabled during the test

Hi, there is no D-gain for yaw axis at all, exept one experimental mode. And it is very bad idea to fly with zeroed D gain on pitch or roll... This was accessible in older versions, but since v.3.5 I've got strong oscillations with 0 D so its impossible to fly.

pidtuner commented 3 years ago

And it is very bad idea to fly with zeroed D gain

Perfect, that's the wait the test should be done, if you watch @bw1129 's video, he mentions:

I am just trying not to hit the walls

These zeroed D gain are just for the "step response" experiment, once we get that blackbox log, the PID Tuner should give the optimal PID gains (including a non-zero D gain for pitch and roll).

Thanks for the info.

pidtuner commented 3 years ago

I have tried to buy from bangood, but none of my cards worked on the site.

I am looking now at aliexpress where I have bought stuff before, but then got confused, because even though all models below support blackbox, they list different firmware:

I am leaning on buying the first one, since is the only one explicitly listing Betaflight, or are all the same? These specs are really confusing for a drone noob.

Appreciate the help

naorunaoru commented 3 years ago

Firmware in this context is the name of Betaflight target.

Betaflight supports a wide range of flight controllers equipped with various sensors and configured differently, so for each unique configuration there's a script which sets up a generic Betaflight installation.

In short, all three of them run Betaflight and all three of them have blackbox, so you're free to pick either one.

mituritsyn commented 3 years ago

@pidtuner you dont need hd version till you gonna buy DJI goggles: https://aliexpress.ru/item/4000829726749.html?&sku_id=12000018678228827

pidtuner commented 3 years ago

Thanks guys, I already ordered the iFlight ProTek25 with Frsky mini XM RX for a very fair price (150eur), it takes a couple of weeks to arrive though.

The BETAFPV transmitter I will buy later from amazon (arrives faster).

Seems I need to watch a lot of tutorials just to be able to setup it up. Its an expensive and complicated hobby you guys have. If you know some setup tutorials for total noobs, I would highly appreciate.

mituritsyn commented 3 years ago

@pidtuner probably this will be the best solution: https://a.aliexpress.com/_mOfeJ3R It has ducts for safety indoor. 128Mb flash for logs, and it comes wo fpv equipment

mituritsyn commented 3 years ago

I already ordered the iFlight ProTek25

My congratulations. Don't forget batteries and charger)

These zeroed D gain are just for the "step response" experiment

I watched the video, found nothing new for myself. I didn't understand where in that video you saw the zeroing of the D component. There's nothing like that. He recommended turning off feed forward and dmin. But nothing about the need to zero D-gain.

pidtuner commented 3 years ago

My congratulations. Don't forget batteries and charger)

OMG this is going to ruin me . What do I need to know about battery? The product specs lists 4S, will this one work? What is the matching charger? I might have to wait till next month to buy transmitter and battery pack, already out of monthly expenses.

mituritsyn commented 3 years ago

@pidtuner charger the battery you need, or any other 4s 650-850mAh and at least 70C from cnhl, gnb, tattu. best prices for cnhl at their official site

pidtuner commented 3 years ago

@mituritsyn thanks, about the zeroing D-gain, I got confused, did not see it in the video but in a blog post:

Find the "The Tuning Process" section, in the post they explain how to tune using a "trial and error" approach, increasing gains gradually and re testing for every new gains combination. My idea would be to tune in one go.

mituritsyn commented 3 years ago

https://www.getfpv.com/learn/fpv-in-depth-and-technical/tuning-your-fpv-drone-with-plasmatree-pid-analyzer/

this is pretty old guide for tool that has no dev activity since 2018

look at this guide, it is for hand tuning but also could be usefull https://drive.google.com/file/d/1WxgSUsG6UnvXjyxJSsAdw9iqoO21Ddmq/view?usp=sharing

pidtuner commented 3 years ago

Well, I am already committed to this

20210610_181059

pidtuner commented 3 years ago

@naorunaoru @mituritsyn maybe you guys can help me out with this one. I bought the Tattu 850mAh 14.8V battery for the iFlight Protek 25.

Now I also bought a TookitRC charger that let's me select Voltage and Amperage to charge each cell. The amps I get, the battery manual says "Do not charge at more than 1C", so if 850mAh then charge max at 0.8A, so I use 0.5A because next one is 1A which would be too much.

But the Voltage I do not get because the battery model says 14.8V, but the manual says:

So what voltage to I charge to? The model name says 14.8V, but the normal range in the manual says max 16.8V?

Even the storage range is larger than the 14.8 from the model name. so I am totally confused. ToolkitRC let's me choose 4.20v, 4.35v, 3.85v and 3.60v. I could use some guidance here, I don't wanna mess up because I read one should be careful with these batteries.

Thanks.

mituritsyn commented 3 years ago

model says 14.8V

li-po has nominal 3.7V so x4=14.8 V, fully charged it will be 4.2x4=16.8 V

Normal Range 3.3V

3.3 is too low try not to discharge it lower than 3.5 or better 3.7

Do not charge at more than 1C

1C is the best for long battery life, but nothing bad should happen if you'll choose 1A for your battery

pidtuner commented 3 years ago

Thanks, then I will charge to 16.8 V when I make the first flights, then 15.4 V for storage. Hope nothing gets damaged,

Everything got delayed because I got the taranis q7 Tx, because I found a lot of tutorials for it, so I thought it might be simpler to setup. In all tutorials I saw it used AA batteries, so I bought high capacity rechargeable. But when mine arrived,it had a slot for a special battery.

So I thought I have to buy one more battery plus one more charger :unamused: but then I found you can actually buy the AA tray for the taranis, but now I have to wait 2-3 more weeks until it arrives :woozy_face:

mituritsyn commented 3 years ago

plus one more charger

you doesn't need it. your charger is universal and can charge all 1-4s li-po. So for q7 you need 2s li-po with xt60 or other plug, but you'll need adapter

pidtuner commented 3 years ago

I have no more budget, so I will wait for the AA tray for the taranis. Still it seems it will not be trivial to get up going according to this guy who has the exact same setup ad I do :face_with_thermometer::

https://www.youtube.com/watch?v=4XBVjnZjrtY

mituritsyn commented 3 years ago

be trivial to get up going according

all should be pretty easy if you can follow the instructions and not blame iflight Instead of frsky as guy from the video)

pidtuner commented 2 years ago

Hey guys, here is some presentation to understand how the PID Tuner works, maybe is helpful. I am developing some algorithms to be able to use later with Betaflight, but first I am testing them on other (easier) platform. Will keep you updated.