Challenge: KK2 vs. PX4

[tubeme]

Hi guys,

I have really deep thoughts lately and cannot understand WHY is that KK2 is controlling our VTOL gradually much better than the PX4?

Our VTOL project started about 2012 when everybody laughed at us at the flying field. This was V1 of the aircraft which was ready in 2013.

https://www.youtube.com/watch?v=Nd8CuSLzw_M

At 2014 we've had a Siniger V2 delta quad with a two patents for it, one for the yaw handling and one for the pitch forward motion. I cannot show pics yet.

We spent most of the time perfecting the air-frame and the aerodynamics and with our two patents we are able to fly the machine rock solid up to 6-7m/s with just a KK2 for the quad and pure manual direct drive for the FW, without any gyro or stab. We are comfortably flying the VTOL in this config in a weather up to 15 m/s. In hover the KK2 is flying the VTOL just amazing in all weather conditions. We have perfect control - forward, backward and yaw self leveling. Up to 5 m/s we are making transition without touching anything except switching on the pusher motor. During the transition the att control is done only by KK2. At higher winds during the transition, sometimes we have to push a little by hand here and there on the roll axis but it is just a minor touch by the pilot.

It took us a very looong way to perfect the frame such that it has almost perfect aerodynamics, so the machine flies ROCK SOLID in terms of aerodynamics and handling.

Sometime in late 2014 we started experimenting with PX4, way before 1.0 release. First we asked a fellow software company to implement the VTOL functionality but during that time you guys made the VTOL so we discarded their work and just started tuning the PX4 for our frame.

We've had zillion PID tuning sessions for the quad and for the FW as well. We've had a zillion tuning sessions for the transition too. All this was acompanied by zillion crashes 1/3 from mechanical and hardware failures, 1/3 from pilot mistakes, 1/3 because of software problems related to PX4 (we were using a lot of master versions)

The result: KK2 + manual FW handles much better than PX4. While FW flight of PX4 is satisfactory the Quad control is terrible compared to KK2.

The obvious observation difference is that KK2 software is assembler vs C for the PX4. But other than that I cannot understand WHY is that so. Why is that this small frail controller handles our VTOL much much better than the mighty Pixhawk + PX4??? Really why?

I read a lot of AHRS discussions, papers and researches etc. I read a lot about the DCM, Mahony, Magdewick, EKF and UKF algorithms and still cannot answer this question...

Given the almost perfect aerodynamics the FW attitude control should a breeze with a very little burden on the hardware. Given KK2 controls the VTOL perfectly in QUAD it should be a breeze to have attitude control with the PX4. But it is not the case with PX4.

Now we concentrated on the 1.1.3 version because want consistent results in a stable version. But still there are problems and problems and problems especially in the transition. We've had 10 crashes for the last week of which 5 are due to bad transition handling in higher winds.

Now we are at the point of producing the VTOL for one day. We've produced more than 100 Sinigers so far, almost like in real production. Also we built one VTOL with our KK2 and fly it in parallel on every flying day in order to get the real feel of the weather and compare it to our PX4 version.

Finally we will make a version that has a passive Pixhawk + PX4 (used as black box) and an active KK2 in order to see the logs and compare the att setpoints that PX4 gives vs. the real handling that KK2 is making.

Any thoughts?

[LorenzMeier]

Since your results are pretty inconsistent with everybody elses results (https://www.youtube.com/watch?v=HorL1iie4YQ&list=PLYy2pGCdhu7xG9NPM7gCsL5-KgaXnyv5_) I'm wondering if its something particular about the way you use the system.

In particular if you are unable to tune pure attitude control in hover mode that should be a warning sign. It is rock solid if tuned right, and you can see that nicely in e.g. Marco's videos. The results from Ada and Simon are less convincing on the attitude control side.

[LorenzMeier]

Can you share your latest log files?

[tubeme]

Hey Lorenz, there are not too many people working on VTOL so there is not a good base for comparison in terms of inconsistency.

And in your particular example it is DEAD CALM weather. In dead calm weather we have good results too. But in this conditions the same is true for our KK2 version without any assistance in FW.

I'm giving myself a warning sign too because we use the KK2 with default settings in the gains. Just flash it with 1.6 version and that's it. No further PID tuning just we put it and fly it. With the PX4 I explained that we made PID tuning from scratch at least 3-4 times. I understand the mechanics of PID tuning, I follow the PID tuning procedures:

https://pixhawk.org/users/fixedwing_pid_tuning

and

https://pixhawk.org/users/multirotor_pid_tuning

And I forgot to mention that we've had a nasty stall rotations that we recover with the flip of a button on the KK2. PX4 does this as well but 1/2 of the times...

I'm not forgetting that we are in the R&D phase and our work is experimental. During our 1 phase of development we've had a zillion crashes with KK2 as well...

I want to be clear - I'm not bashing on the PX4 but rather I want to understand where is my mistake.

I will post the LOG's after I select some of them from the last week. The crashes from last week are due to high winds, low transition speed set in the parameters, and quad motors stopping almost imminently and not getting this 3-4 seconds in transition push until the quad stops.

[tubeme]

Also I forgot to mention that we are flying almost every day at least once. Sometimes twice a day in California in all kinds of weather. Most people here I see use SIMs to detect problems. We have a hands on approach. Call it stupid but that is our way of doing things. We fly everything not simulating.

[LorenzMeier]

If you are flying that often then I don't understand why you are not sharing more logs or participate more in the flight testing of the current software development. You seem to loose a lot of airframes, but at the same time the development community benefits very little from it in gaining insights. I have not seen you involved with testing pull requests / fixes yet - did I miss that?

That seems mutually unproductive. I also should highlight that all code that went into master was extensively flight tested, including high wind conditions. It sounds like you could be contributing a lot, we just need to find a way to make that more efficient and more productive. Would you consider starting to flight test the development state more regularly?

I have the impression that you're staying on stable / releases with your testing. Which is a mistake for VTOL, because you will crash because of things that have been fixed for a long time in the current VTOL code. You should abandon that approach as soon as possible and instead get in touch with the VTOL developers and make sure you always use what they recommend - which changes daily.

@sanderux @tumbili @AndreasAntener FYI

[sanderux]

@tubeme i would note that your solid results with manual control would be identical with the px4 if you use the fc only for quad. I can testify to this as i have tested this myself.

I had varying results during development, but most were related to bad tuning. Most recently i changed a param related to max altitude velocity for mc and wham, rock solid transitions. All i need now is some tuning on fw to get solid altitude holding. The failing params were identified because i share all my logs. Without that i would be stuck with a very bad performing machine.

My build is online including a video build log and i publish and maintain the model actively. I also envision a market and I will be producing commercial VTOL models, but i don't keep everything hidden and private because i do not believe that anyone with the capability to build one would ever buy one. Whenever i find something that could be better i change it, either by doing it myself or by working with the other developers. I believe that attitude will move the entire market forward and that would benefit me much more then having a unique but unsupported model for a few months.

I fully agree with Lorenz in;

• the observation that you do not seem to share logs that would be very helpful, especially when you fly every day. We are desperate for flight testing because a lot of VTOL changes are hard to test in simulations.
• Sticking to stable versions is usually a good idea but not for VTOL. The current master flies much better then the release versions.
• You appear to understand the architecture of the code, if you have the ability to identify flaws in the code it would help a lot if you submit PR's or at least suggest changes. I would personally be more then willing to setup branches with your suggested changes for you to test.

So if you experience problems please;

• Fly the current master and see if it is solved
• Share your logs and notify us of any problems, we can also help you if it is the result of tuning
• If you can, help us getting the best and most stable VTOL platform, it will benefit you and your endeavor a lot more then keeping everything closed.

[LorenzMeier]

@sanderux Have you already updated the params on master with your latest tuning results for the quad ranger?

[sanderux]

No, waiting on one more change to test this friday and will update to a fully stable flight and transition.

[sanderux]

@tubeme if you share one log with your current params i will be able to verify with my params, your models should be quite similar to mine.

[SimonWilks]

Hi tuberne

"Why is that this small frail controller handles our VTOL much much better than the mighty Pixhawk...". You should know that size isn't what counts, it's how it's used ;) What I mean by that is, of course, it's probably the software.

I haven't used the KK2 but I've heard pretty good things about it and that it has supported VTOL for quite a while so I could imagine that a lot of work on the software side, testing, tuning, etc. has gone into it over the years. On the PX4 side a lot was done last summer and then we had to drop it for a long time due to other commitments but now we are all back on it quite intensively. It is now just reached the point where it flies quite well (even in very windy conditions where forwards flight was hard to achieve but transitions were still quite good) but only IF you have everything well tuned. I would also say that knowing the way it's built gives us a certain advantage over anyone else still. But we are working hard to get it there. And we will.

I can understand your frustration about why VTOL on PX4 doesn't just work, but everything has to start somewhere, run through it's development cycle, get tested, improve, evolve, etc. until it reaches a level that one would consider it rock solid along with great documentation. That takes time. I assume KK2 didn't have this overnight either.

If you want PX4 VTOL to work as well or better than KK2 I think what would be most helpful for us (and very welcomed) is if you would provide us with data...in various forms. Give us videos, descriptions/comparisons and more importantly log files. Really important is....yes, log files. This is pretty much a prerequisite for getting any help. I don't know if KK2 can do it (I assume it can) but you could also consider giving us screenshots of plots from flights you've done with that system in multirotor, fixed-wing and/or VTOL. Take a look at some of Mark's recent posts comparing flight with the Pixracer and Naze. That's exactly what's needed.

If you are suffering with your product development because of this you should get directly/actively involved. Contact us, help us and we might be able to help you. We also need testers to help move things forwards quicker and provide direct support to get us there. Others are doing this and are learning a LOT, helping us a LOT and we are all benefiting. If you just silently take what is appearing in master you really are a bit on your own.

Contrary to popular belief, open-source is often not free, especially if you want to be on the leading edge. The price is often helping get through periods like this and actively contributing otherwise you'll just be on the backburner.

Let us know how we can help each other.

Regards Simon

Co-founder | UAVenture AG http://www.uaventure.com simon@uaventure.com andreas@uaventure.com +41 (0)62 544 93 12

Aarenaustrasse 31 5000 Aarau Switzerland

On Tue, Feb 23, 2016 at 9:25 PM, Sander Smeets notifications@github.com wrote:

@tubeme https://github.com/tubeme if you share one log with your current params i will be able to verify with my params, your models should be quite similar to mine.

— Reply to this email directly or view it on GitHub https://github.com/PX4/Firmware/issues/3832#issuecomment-187887621.

[tubeme]

@LorenzMeier @sanderux @SimonWilks I totally agree with you guys, but there are a couple of reasons for this. At first we were not familiar with the PX4 then with time we got decent knowledge. So with time we moved from the discussion forums to github. Sometimes our issues posted in the discussion forums got neglected by the dev team so we felt our questions or reports are burden for the core team. And we try to have as little footprint in the github and post really fundamental puzzling questions for us in certain flights that we cannot understand. Lastly about a month or two ago I sent Lorenz a good diligent report, about the transitions, on which we did not get any reply. Again we thought that our feedback and reporting does not interest you. The report was done with graphics, logs and some analysis from me , it covered almost an year of flying in 4 cases and was sent by e-mail.

We will gladly take more involvement with the testing and conceptualizing and giving ideas. We have decent knowledge in many areas and aspects of the aircraft building and flying. In Bulgaria we have a very good machinery production facility with CNC Milling machines and other machinery equipment. We make our own Helicopters and mechanics for them. In this facility we produce 50% of the parts for a German turbine engine brand (model turbines). So we are planning on building Turbine plane soon, we've had it ready on 70%. In California we have a 3D CNC cutter for EPS and EPO so we can produce complex wings with twists etc. Also there is the Composite workshop and the VTOL production.

I don't feel comfortable writing code despite I'm B.Sc. Comp.Sci. I graduated 99 but was working in the Marketing Advertising TV business since then and my skills are very rusty. I need about 1-2 years to get in to shape, but I prefer to do what I do best, make a product and sell it. So we are concentration on this. I have a general idea of the code and the architecture but not in depth one. With time this will change and I will get more in to the code. My strengths are connecting the dots, management and concepts so we try to stay focused and release some products and get sustainable business model that can attract investment in order to be able to continue development. We all ready work on Siniger V3 :)

I'm a programmer by education so I respect the code! That is why I did not create pull requests or get in the code details.

I will post more Logs and more cases.

Best Vasil

[tubeme]

[LorenzMeier]

@tubeme Nice airframe! My apologies for not being always able to respond in depth to logs. Some times of the year I'm completely overloaded, some times there is room to spend time on specific issues. The best way to solve this is coordination ahead of time: Use Github, the dev call or the mailing list to find out when people have time to focus on VTOL development and try to synchronise your own efforts to it. If you do, you can get extremely low response times and great interest.

As Simon said developers also find log files much, much more interesting when they get those for the current software they are working on. That's a bit different for multicopters where things have been very stable, but for VTOL the codebase of last week looks very different from the one this week.

So lets try to get this process started: Here are the currently open issues for VTOL: https://github.com/PX4/Firmware/milestones/VTOL%20Enhancements%20for%20v1.3.0

Feel free to file new ones with a log file attached. Try to point to specific problems you had (e.g. stall on forward transition).

@AndreasAntener @SimonWilks @sanderux @tumbili Probably can also ping you when there is a new PR ready to test, and if you join the testing regularly the whole development will happen a lot faster.

[tubeme]

@LorenzMeier NP for the report. We find it much more productive to make new issues here, describe the problem and cover it with Logs etc.. Please ping me for PR's related to VTOL. We are going to participate in the dev call as well. I just have to setup the thing with Mumble.

If you allow me I can give ideas and give my input on usability. I have pretty decent experience with other flight controllers as well.

[sanderux]

@tubeme you can follow all issues and PR's on VTOL by following the VTOL label. https://github.com/PX4/Firmware/pulls?q=is%3Aopen+is%3Apr+label%3Avtol

[LorenzMeier]

Closing as stale - VTOL improved a lot since and I think the discussion here revolved around manual flying / tuning params / airspeed sensing. Tuning is airframe specific and we support now VTOL without airspeed sensors.