Question: Development workflow

[simonseyer]

Hi SkyDrop devs!

First of all: thank you for open sourcing the firmware. That’s well appreciated!

Disclaimer: I'm an embedded development noob (but have an iOS development background).

I would like to add some functionality to my SkyDrop to configure it via BLE as well as fetch track files and install airspaces. I've found that you've started to implement a skybean BLE protocol and thought I pick up on that.

I was able to build the firmware using the cmake PR (with a few adjustments necessary, probably because I'm on macOS), install it and got the firmware to freeze as soon as it initialised BLE (at least that's how it looked like). After being in shock for a bit because I bricked my new SkyDrop (😬) I figured out how to recover it by putting a UPDATE.FW directly on the SD card (RTFM).

Now, however, I'm wondering if there isn't a better development workflow. It's obviously painfully slow to install the firmware via the SD card and I can't debug my code. I assume that by doing some soldering and connecting the device to a hardware debugger I could solve both problems but I'm not only an embedded noob but also an electronics noob, so I don't want to destroy my new SkyDrop.

I've spent my Saturday looking into different chip simulators. The farthest I've got with MPLAB X using the prebuilt .hex file but when I open the IO Viewer it crashes the IDE. I also tried to configuring the MPLAB X to build the firmware from source but wasn't successful either.

Are you by chance be able to point me in better direction here? Is this idea of mine to extend the firmware without hardware debugger even realistically feasible? All help is appreciated 🙏

Also tagging you @bubeck and @hiaselhans here since you have been active on the project last. Hope that's ok :)

[bubeck]

Hi Simon

Thanks for asking and thanks for taking your time to improve SkyDrop.

I also tried simulation, but the device is too specific to find a good solution, so I gave up on this. But if YOU find a good simulator, then please let us know. Things are changing over time, so maybe now is something available.

There are some challenges:

• The PROGMEM is nearly 99% full, this means, that you probably have to remove existing functions in order to improve. There is a NEED_LOT_OF_SPACE_FOR_DEBUG in debug.h which can be used as a starting point. I was thinking about a modular firmware generator, where a user can select which function he needs, and then the correct #defines will be generated and a firmware produced. But this is lot of work to do.
• development attention is now on SkyBean Strato, therefore it probably takes some time for your functions to get included.

I always used DEBUG macro to print out, what is going on. But this is also challenging, as it first gets cached in memory and then written out from time to time. If there is a crash, you might miss the last lines....

You should normally not need the SD card out of the device. Typically you use a USB cable to transfer SKYDROP.FW to the device (onto the SD), without removing it. Yes this takes some time. :(

The main author is fhorinek, but he is very busy on strato. Feel free to ask me any questions, I will help as good as possible. However, I never touched bluetooth.

[simonseyer]

Hi Tilmann,

thanks a lot for your reply! That's all very helpful.

I also tried simulation, but the device is too specific to find a good solution

One thing that I've recognised is that the atxmega series is not as well supported like for example the atmega series in terms of simulation. The other limiting factor is my platform of choice. I think I'll setup a Windows VM and will tinker around some more. I'm not too hopeful though.

The PROGMEM is nearly 99% full

That's a good hint and a bit of a bummer. I will see how far I can get without touching too much of the codebase.

development attention is now on SkyBean Strato

Also given the program size limitations you've mentioned, it would actually be great to maybe have a v2 of the SkyDrop at some point. I really like the form factor, because it provides a small, lightweight, robust, standalone solution that I can easily combine it with my smartphone for longer flights having best of both worlds (large high resolution/frequency display for things like navigation, thermaling assistant + the sensors of the SkyDrop).

If the v2 would additionally have FLARM/FANET support an upgrade would be a no-brainer for me. The Strato doesn't appeal to me very much because the display just can't reach a smartphone one, also in terms of usability off-flight (touch). What I also really like about the smartphone solution is the proximity sensor that allows me to switch views without having to press a button.

therefore it probably takes some time for your functions to get included.

So far I would be fine with working on a fork. I don't want to commit to anything because my spare time is limited, but maybe we could at some point have a "community" version of the firmware. I was also thinking of implementing a firmware update via BLE. If that could become part of an official firmware release, it would be very easy to distribute "community" version as well. Just thinking out loud here :)

I always used DEBUG macro to print out, what is going on. But this is also challenging, as it first gets cached in memory and then written out from time to time. If there is a crash, you might miss the last lines.... You should normally not need the SD card out of the device. Typically you use a USB cable to transfer SKYDROP.FW to the device (onto the SD), without removing it. Yes this takes some time. :(

In my specific case I've introduced a bug that also took the USB mode and the visual firmware update with it (since they are part of the application unfortunately). In USB mode it didn't crash but it wasn't communicating with the PC either (showed a flat line). So I wasn't able to recover it without disassembling the device.

I'm actually still not sure what caused this crash (and have no good way of finding it out). My guess is either of the three:

1. https://github.com/fhorinek/SkyDrop/pull/447#discussion_r775651989 — I've "inherited" this change because I used the cmake branch — I've reverted it in the meantime but didn't try again yet. Maybe this corrupted the eeprom somehow 🤔
2. https://github.com/fhorinek/SkyDrop/blob/master/skydrop/src/fc/protocols/skybean/skybean.cpp#L58 — protocol_skybean_init is the only code that I've compiled in that wasn't executed before (in particular after BLE got initialised, which seemed to be the time when the screen froze)
3. I messed something up with the compilation which somehow resulted in a corruption — I think that's the most unlikely option since it booted correctly and actually worked the first time (including me selecting the new "skybean" protocol in the settings)

I think I will give it another shot without the changes in 1. and see if it shows the same behaviour again. If it indeed crashes, I would create a branch with my changes and would appreciate very much, if you could maybe have a look at it.

Another thought of mine was that it would be great if the USB mode would be more independent of the rest of the application to avoid such traps in the future. I'm surely not knowledgable enough in firmware development to achieve this anytime soon though.

Last but not least I was wondering why it takes so long for the SD card to even be even shown on my system. Is the same for you?

The main author is fhorinek, but he is very busy on strato. Feel free to ask me any questions, I will help as good as possible. However, I never touched bluetooth.

That's very kind of you! Thank you very much. I've actually got quite a bit of experience with Bluetooth, not from an embedded but a mobile perspective. Maybe this helps :)

[bubeck]

Also given the program size limitations you've mentioned, it would actually be great to maybe have a v2 of the SkyDrop at some point. I really like the form factor, because it provides a small, lightweight, robust, standalone solution that I can easily combine it with my smartphone for longer flights having best of both worlds (large high resolution/frequency display for things like navigation, thermaling assistant + the sensors of the SkyDrop).

I am not connected to SkyBean so this is no official statement, but my private opinion: I dont think, that there will be a SkyDrop v2, as the strato is meant as the successor.

One thing that I've recognised is that the atxmega series is not as well supported like for example the atmega series in terms of simulation. The other limiting factor is my platform of choice. I think I'll setup a Windows VM and will tinker around some more. I'm not too hopeful though.

Also a VM with Windows might work, but the development of SkyDrop (and also Strato) takes place in Linux. So maybe Linux would be a better choice.

Make sure, that you have the right compiler version installed. I am using avr-gcc-7.4.0 which is OK. Any newer version seems to product much bigger codebase for a unknown reason. I would propose to start with a clean fork and just recompile without any changes. As long as this is not working, I would not start changing anything. cmake should be OK, however, eclipse with avr-plugin is recommended.

Last but not least I was wondering why it takes so long for the SD card to even be even shown on my system. Is the same for you?

How long does it take? Connecting SkyDrop in USB mode to Linux takes around 20s which is needed as the processor is not very fast. Is there a difference between your compilation and the original firmware? There should be none.

[simonseyer]

I am not connected to SkyBean so this is no official statement, but my private opinion: I dont think, that there will be a SkyDrop v2, as the strato is meant as the successor.

Thanks for your assessment. Would be understandable, it's quite a lot of effort to maintain multiple devices. And the market is relatively small.

Also a VM with Windows might work, but the development of SkyDrop (and also Strato) takes place in Linux. So maybe Linux would be a better choice.

For the most part macOS and Linux are comparable for command line tools. At least based on my experience, but the devil is in the details of course. I was also able to compile and run a Linux-only simulation tool on macOS with a few minor adjustments for example. The reason I was looking into Windows were tools like AVR Studio which are Windows-only.

Make sure, that you have the right compiler version installed. I am using avr-gcc-7.4.0 which is OK. Any newer version seems to product much bigger codebase for a unknown reason.

Got avr-gcc (GCC) 7.2.0 installed using the avr-toolchain/build.sh script.

I would propose to start with a clean fork and just recompile without any changes. As long as this is not working, I would not start changing anything.

Yeah, that milestone I've passed. Which indeed removes point 3. from the list of potential problems, good point.

cmake should be OK, however, eclipse with avr-plugin is recommended.

Unfortunately Eclipse on my machine doesn't even start anymore after installing the plugin. Installing an old Eclipse version was also not straightforward. cmake generally seems to work™️ though.

How long does it take? Connecting SkyDrop in USB mode to Linux takes around 20s which is needed as the processor is not very fast. Is there a difference between your compilation and the original firmware? There should be none.

Roughly the same time for me. No difference between firmware versions.

[bubeck]

Unfortunately Eclipse on my machine doesn't even start anymore after installing the plugin. Installing an old Eclipse version was also not straightforward. cmake generally seems to worktm though.

cmake patch has also modification on eeprom_read/write. Maybe the cause of your problems. Try to use cmake changes only and nothing else.

[simonseyer]

Yeah, that's my plan 👍

[simonseyer]

It's working now 🎉 The eeprom_read/write modifications were probably the issue indeed. I've also ran into he maximum program size limitation and used the NEED_LOT_OF_SPACE_FOR_DEBUG to get around it for now.

If you are curious, here's the diff: https://github.com/fhorinek/SkyDrop/compare/master...simonseyer:skybean-protocol

Since macOS doesn't ship with python2 anymore since recently I've migrated the build scripts over to python3 which seems to work fine after a few modifications.

Now I can try to exchange some information via the existing SkyBean BLE protocol and start to extend it. Thank you for your help so far 🙏 I'm sure I'll need some more support in the future.

[bubeck]

Since macOS doesn't ship with python2 anymore since recently I've migrated the build scripts over to python3 which seems to work fine after a few modifications.

Could you please issue a PR for this?

[simonseyer]

Sure, happy to do so: https://github.com/fhorinek/SkyDrop/pull/464

[fhorinek]

Hi Simon,

I am happy that you are interested to add new features to the firmware.

However, @bubeck is right, SkyDrop firmware development is done.

Main reason is that the memory is full. I needed to re-optimize old code to fit few new lines multiple times. (Not fun) Also we needed to try every avr-gcc version to have smallest code possible.

Next reason is that the hardware is getting old and the components are hardener to get every year (chip shortage does not help). There won't be SkyDrop v2. But there is something else, already in development. It won't be direct replacement of SkyDrop but it will have fanet/flarm combo with ble and much more modern mcu. This is still in early stage, so I won't give you much more details.

The focus now is on the Strato, and there is still lot to do there. The form-factor is not smartphone like for many reasons (making smartphone like device is hard, surprisingly) The display is not as nice as on smartphones, but it is trans-reflective, designed to be visible without backlight on direct sun light! Also the reason to not include touch is to not obscure the lcd by additional layer on the screen.

Few random informations: Slow USB on SkyDrop: device is slow, it even can't keep with the USB clock since the MCU frequency is 32MHz, while USB is running at 48MHz. MCU is still behind, missing the frames, and that is causing delays.

Simulator: There is no (usable) simulators for xmegas, they are dead branch since now there are ARMs, tesilicas and risc-Vs. Community is not doing much to support them, because of small market share. And since Atmel was absorbed into Microchip it is even worst.

Debug: You can use PDI enabled debuger like dragon and avarice to debug your code, but you can't fit the debug firmware version to the MCU, because of the additional meta information that you need to use GDB. Also it is painfully slow and you will need to solder the connector.

Bluetooth: there was an idea to do what you want, look here: https://github.com/fhorinek/SkyDrop/tree/master/skydrop_bt_client This is command line interface, that enables you to manipulate files on the sdcard via bluetooth. But it require the skybean protocol, which was cut from the code, because of the size. We also do not have enough resources to support mobile app.

Modular firmware: Great Idea, but now I have to be business man as well. Not many users even change the defaults settings on the SkyDrop. How much impact to our customer base, would have the ability to customize the firmware? Few of them would even notice it. The added maintenance cost would not be justifiable.

Bonus: If you gone this far :-) I have proposition for you, use your ideas on Strato. The debugging is much easier with modern MCU and fast stepping. You will get familiar with embedded systems and your contribution will be able to go to the mainstream much easier. If you are interested, I can supply you with the Strato and dev kit for free. If you don't like them, and don't want to work on them, just send them back, no strings attached.

Later when the SkyDrop "replacement" will be ready, you will be able to hack it from day one, I will even send you one, if you will be active with Strato :-) Just please don't waste your enthusiasm on the SkyDrop. It can be very frustrating experience with the memory constraints. Trust me I know.

Fero

[simonseyer]

Hey Fero,

first of all thank you for your extensive answer, sharing your perspective and taking the time to answer my questions.

The SkyDrop replacement sounds interesting and I'm curious to hear more about it in the future.

The size limitation was indeed on my mind as well and while I was pushing on for the sake of learning something and "proving" that I can make the communication work, I already accepted that this is probably not a something that I will be able to finish. Adding the outdated MCU (that I wasn't aware of) and having reached the milestone of reading out the device name via your protocol in my iOS app, I guess it's a good time to stop now.

This was fun though! So I would be happy to accept your (very!) kind offer of providing me with a Strato. I can't promise anything (but you didn't ask for that :)), in particular over the course of this year, but I would be happy to give it a shot. I also promise that I'll take better care of it than I did with my new SkyDrop 😬 (Denisa knows).

Thank you again Fero!

Best

Simon