Add ability to configure BLHeli ESCs through GCS

[mluessi]

It would be nice to be able to program BLHeli (or other ESCs) through the GCS without having to remove wires from the flight controller. The following is needed to make this work:

1. A software serial driver that allows communication with the ESC using a GPIO pin (the pin that is normally used for PWM output). The driver must also be able to switch between PWM and serial mode (ideally without requiring a restart).
2. A way for the driver to send serial data to / from the GCS. The easiest would be to use an UAVO for this.
3. ESC configuration in the GCS. The easiest would be for the GCS to communicate with BLHeliSuite, as it is done in cleanflight. However, BLHeliSuite is kind of crappy and Windows only, so it would be better to implement this natively in the GCS.

[dustin]

blhelisuite burned at least an hour of my last build and basically caused me to give up for a couple nights (I got it working eventually).

They seem really not willing to make something that isn't windows work, which is sad because it appears to be a fairly simple program. I'm surely not the only person who really hates using blheli just because of how hard they make it. Making it easier to use their products seems like it would be in their best interest.

Configuration read and write (which would include identification) should be quite simple. Firmware updates might be harder. With a little bit of protocol definition, that might not be the case.

[tracernz]

Soft UART driver: https://github.com/peabody124/TauLabs/blob/softusart/flight/PiOS/STM32F10x/pios_softusart.c

I was thinking of implementing some kind of mechanism whereby a module can acquire a lock to prevent arming, and all locks must be returned before arming can occur. I think this would be a good improvment in general for configuration of the controller and probably necessary if we are going to switch the servo driver for a soft uart driver, thoughts?

[WZ9V]

I've been digging a little and as best I can tell CleanFlight has a MSP extension MSP_SET_1WIRE that switches all of the ESC signal lines to pass through (soft UART) mode. Which then makes the FC just a fancy programmer box for the BLHeliSuite software. I'm still working my way around the d-Ronin code but I was thinking of making an attempt at the pass through capability. If we could get the FC to act as a programmer box that's at least a good first step. BTW they made progress in getting the Windows software working under WINE. I have the basics going and can talk to my Arduino Uno board but I have to make an adapter for the ESC cables to test out the whole thing. As far as variable mapping, once we have the FC acting as a pass through we could packet capture what the Windows program is sending a reverse engineer how to set the variables. We could then create some sort of XML definition for the variables and needed revision level that the GCS could use to display a configuration screen.

The trick here is finding the time to work on this.

[dustin]

Reverse engineering a particular version is not good for us. We need a stable interface that blheli would commit to. It's unlikely they would do that since it restricts what types of things they can do.

[tracernz]

I have done work lately on CF code adding real passthrough with softserial rather than the current hacky implementation. I opened an issue yesterday to move the MSP encoder/decoder into a library. There is a soft-serial driver in one of peabody's branches. What remains is the infrastructure so we can obtain a lock on arming of some sort, and disable PiOS_Servo so it doesn't interfere. Then work can commence on implementing this.

[WZ9V]

Cool. Sounds like you all are already on it. Pass through and using their GUI is fine by me. I do believe that a flexible way of displaying a configuration screen could be worked out though. While not having to reverse engineer their changes would be better, it wouldn't be that hard to have a definition DB that defines what a particular version expects, query the interface to get the version and build that screen dynamically for the GCS (at least at a conceptual level). I'll do some experimenting along those lines using my Arduino interface outsidevof d-Ronin and see where I get. At the very least I might manage to make a Linux native GUI ;-)

[mlyle]

We're certainly thinking about it at least, but it's a little while coming. Any and all help appreciated :)

One of the trickiest choices to make is the encapsulation, etc. There are some reasons that encapsulating serial in uavtalk would be nice-- it would work over radio interfaces, and it could be used for other things, too-- like differential GPS corrections from a ground station in flight.

In that case, GCS needs to provide a virtual serial port that blheli, etc, can talk to-- which there's a degree of magic involved in.

Alternatively, we can just use normal VCP for this, and put all of the work onto the flight board.

[tracernz]

I think VCP at first makes this a lot closer. Getting BLHeliSuite talking serial to GCS for Serial-over-UAVO is likely to be problematic in itself IMO.

[WZ9V]

I was planning to do some work on a Python or whatever program to talk to the Arduino programmer. I'll work on that and maybe that could help since we wouldn't have to worry about WINE and Windows. Rather than Python I'll look into using Qt instead. I'm more familiar (but rusty) with C++ anyway. Using Python was just an excuse to practice to get more familiar with it. Won't make any guarantees (been management for quite some time) but I'm interested in working on this. I'd love to tune/tweak the ESC without having to undo wiring.

By VCP you are talking the /dev/ttyACM(X) device correct? In the Arduino case it's showing up as /dev/ACM0 on my laptop. I have the 4-wire document so I'll start playing around with getting a Qt program to query the version number from the ESC using the Arduino box and display it.

[tracernz]

I was planning to use the MSP commands BLHeliSuite already supports so the entire thing can be controlled by BLHeliSuite, including selecting different ESCs etc., no GCS support required (other than configuring the VCP port and enabling the module initially.

[mlyle]

re:VCP. Yah, I don't think better should be the enemy of good. I wouldn't build it that way, but i wouldn't object to that implementation merging either.

On Fri, Feb 19, 2016 at 8:13 PM Michael Corcoran notifications@github.com wrote:

I was planning to use the MSP commands BLHeliSuite already supports so the entire thing can be controlled by BLHeliSuite, including selecting different ESCs etc., no GCS support required (other than configuring the VCP port and enabling the module initially.

— Reply to this email directly or view it on GitHub https://github.com/d-ronin/dRonin/issues/104#issuecomment-186502671.

[tracernz]

Yep, ultimately we should do the UAVO-serial thing but not at the cost of blocking other stuff.

[mluessi]

I think this is a good plan. BLHeli suite passthrough should be achievable for Tanto, after that, we can start implementing direct configuration using the GCS. Not having to use VCP would definitely be a benefit, as many Windows users seem to have driver issues.

@WZ9V keep us posted what you find. Once we know the protocol and differences between versions it should be fairly easy to add the GUI to the GCS :).

[tracernz]

@ mluessi About half our windows users are on Windows 10, and this is increasing. VCP works out of the box there so it shouldn't be too much of an issue.

[WZ9V]

You are thinking exactly along the lines I was. Let's aim for the simple pass through first and over time morph it into a fully integrated d-Ronin GCS implementation. The only part I have not been able to dig up is how the variables on the screen are layed out in the EEPROM. I'll try asking him for a mapping, if that doesn't work I'll resort to a serial capture and spend some time staring at the hex values. Wouldn't be the first time I had to resort to that.

[tracernz]

You will find that in the BLHeli source code.

With some small extensions to the soft serial driver SimonK bootloader through BLHeliSuite can also be supported.

[WZ9V]

Thanks, I'll check there. I still hadn't found that information yet.

[tracernz]

@WZ9V Atmel EEPROM layout: https://github.com/bitdump/BLHeli/blob/master/Atmel/BLHeli.asm#L1277, SiLabs https://github.com/bitdump/BLHeli/blob/master/SiLabs/BLHeli.asm#L2259

[WZ9V]

Thanks. I'm still fighting with sending a command and receiving a response via /dev/ttyACM0 to the Arduino programming box. I was messing around with it in Python with PySerial but I think I might be better off in C/C++ which I understand better. Those layouts will be handy once I get some basics working.

[tracernz]

Are you writing a BLHeliSuite replacement?

[WZ9V]

Ultimately that's what I'm aiming for. For right now I'm just experimenting. Python and PySerial are not cooperating at the moment. I have a dummy program on the Arduino that just prints a string on the LCD. It works from the serial monitor but my super simple Python script says it sends the 12 characters in my test string but the Arduino is not receiving them. I'm essentially still at the "Hello World" stage right now. Switching to the Qt Terminal example now since I'm getting nowhere with Python and PySerial.

Really wished I hadn't wasted all afternoon with Python. Got the C++/Qt Example going shortly after downloading the files. Now I can reload the the BLHeli stuff and go back to the real experiment.

[WZ9V]

Little bit of progress, got my Qt Terminal example to send the keep alive and get the correct response. Turns out there was a missing command in the docs send_InterfaceListen - FF that tells the interface to switch from the GUI mode to PC mode. Bit Rate is 38400, N,8,1 when talking PC to interface. Setting up repository for further work on Qt based BLHeli Client.

[Tomcatt]

Hello all. What's the status on this little project?

[WZ9V]

I haven't made much progress on my Qt port mostly due to lack of time to work on it. I did get the basic Qt terminal sample to talk to a BLHeli interface board to query the version on the ESC.

[faalbers]

Thanks for all the great research work on BlHeli passthrough ! ;)

[mluessi]

As far as I know, the most recent betaflight configurator supposedly has options for blheli programming built-in, so we should be able to get some information on the protocol from there. I looked through the code but didn't see anything. Does anyone know more about this?

The other missing piece is a bi-directional softserial interface. If we build the programming into the GCS, it is only necessary to communicate with one ESC at a time (the GCS can do the sequencing between ESCs). So from what I can see, we need a way to tell the output module to stop PWM output of an output (probably the entire bank of outputs, so we can use the timer for input and output) and then the user connects the battery. Some STM32 softserial code that uses a capture / compare channel and interrupts here: http://www.stm32duino.com/viewtopic.php?t=864 this may be a good starting point.

For sending ESC data between the GCS and the FC, we could add two UAVOs, e.g., ESCDataTx and ESCDataRx, that has some basic information (which ESC to talk to, data length, and data field). So the code in the flight controller could be quite minimal.

I hope I will find some time to look into this soon. Any tips, links to resources, etc are appreciated.

[tracernz]

My concern with doing the actual flashing/configuration in the GCS is the maintenance required. The BLHeli release cycle is much shorter than ours. We would need to maintain EEPROM and Flash layouts for each version of BLHeli, and for each ESC variant. I'd rather we just support BLHeliSuite and leave that burden to them, which works well because their release cycle is synchronised with BLHeli. I have a module which speaks BLHeliSuite's 4W-IF protocol through PIOS_COM (so it can work over a physical serial port or VCP, VCP being the obvious choice). I don't have the ESC protocol side though, which would need a soft-serial driver and some way to prevent arming, and disable PWM output on the channels. This is the way cleanflight now works and the protocols are documented through the source code @4712 merged into CF. What do you think?

P.S. If we do serial-over-UAVO we should make it totally generic and a PIOS_COM driver so it can be used for other things as well.

[mlyle]

Yes, I want serial-over-UAVO for other things. I prefer the serial-over-uavo approach, but there are problems-- really need a "fake serial driver" for it to work with blheli configurator.

There's things like this that could work-- http://com0com.cvs.sourceforge.net/viewvc/com0com/com2tcp/

[mlyle]

Also to note-- the mechanism that disables/enables servos... we should think about how it will work with ESC telemetry--e.g. castle link telemetry-- at the same time. It would be unfortunate to have two different competing mechanisms fighting each other.

[tracernz]

but there are problems-- really need a "fake serial driver" for it to work with blheli configurator.

That's why I went for VCP for at least initial impl. It works without any fancy os-specific stuff.

[mlyle]

@tracernz I wonder if we could just write the functionality for blhelisuite and get it accepted. (TCP/pipe)

[tracernz]

BLHeliSuite is closed source sadly, and written in Delphi.

[mlyle]

Hm. Maybe we can beg them to support it.

[tracernz]

Yah, no point blocking on that though when it's easy to get it going with VCP already.

[mluessi]

@tracernz good point about the pain of needing to maintain EEPROM / Flash layouts; I somehow assumed all blheli ESCs speak the same protocol and low level things like this gets handled in the ESC, but if it doesn't we definitely don't want to have to maintain that.

I like the TCP/IP interface idea. It should give a better user experience as we won't have the VCP driver issues that seem to be frequent on Windows.

[mluessi]

@tracernz Can you put the code you have in a PR, even if it is non-functional. It's probably a good idea to start working on this using VCP and then switch over to TCP/IP if/when blheli suite adds this feature.

[mluessi]

@tracernz any luck locating the code?

[4712]

@mluessi Yes please VCP first, TCP needs some more time...

[mlyle]

@4712 thanks very much for the info and your help.

[tracernz]

@mluessi I sent you a PM on IRC a few days ago. 😛 Unfinished, but it worked quite well up to the point BLHeliSuite tried to connect to individual ESCs. https://github.com/tracernz/dronin/tree/mrc-blheli Protocol spec is in @4712's repo, and the BlHeli bootloader protocol is probably best documented by his code https://github.com/cleanflight/cleanflight/pulls?utf8=✓&q=author%3A4712, or there is some German language documentation on Google (it's called AVRootLoader). STK500v2 protocol (used by SimonK) is documented by Atmel (there are many STK500 protocol docs available).

@4712 Nice to see you here. 😄

[ufoDziner]

After having to mess with ESC settings on my latest build, I can certainly see the desire to have this done. @mluessi Is this getting close? Thanks!

[tracernz]

Closing this since we can now do this through BLHeliSuite. I don't think we ever want to take on the maintenance burden of BLHeli configuration ourselves.