Possible to pull MicroPython runtime from device, update, and then flash?

[knowledgejunkie]

Is it possible for uflash to pull a runtime from a micro:bit, validate that it is a MicroPython runtime, and then reflash the device after inserting the desired Python scripts, or is the process write-only?

The reason for the query is that the MicroPython runtime, as built via yotta and distributed with uflash/Mu, is built partly (as far I can see following the yotta build dependency chain) from sources that include non-free code (https://github.com/ARMmbed/ble-nrf51822 for example). Please correct me if this is not the case.

If it were possible to pull the runtime off a device prior to flashing, it would mean that the firmware would not need to be distributed in the downstream packages of operating systems that only distribute Free Software.

The MIT-licensed uflash repository contains firmware.hex, and uflash.py contains a copy of the firmware, and is MIT-licensed per header. Can I check that this is compatible with the more restrictive licensing of some of the upstream source(s)?

(I note the separate issue discussing the provenance of the included firmware.hex file at https://github.com/ntoll/uflash/issues/46).

[ntoll]

Hi @knowledgejunkie,

First of all, thank you for taking the time to bring this up. It's an angle I hadn't even considered. I'm going to cc @jaustin who is both familiar with this project and also work[s|ed] for ARM in the group responsible for yotta. If he can't clarify the situation, he'll probably know someone at ARM who would be able to advise.

@jaustin, the question is if it's OK for the firmware for MicroPython on the micro:bit (built, in part, using yotta which is, apparently, non-free) to be distributed in an MIT licensed project like uflash. I.e. as @knowledgejunkie says, we want to check such a situation is compatible with the more restrictive licensing of some of the upstream source (yotta).

IANAL, but I'm thankful people like you folks are looking at this.

[carlosperate]

Hi all,

I can probably clarify a few points.

Is it possible for uflash to pull a runtime from a micro:bit, validate that it is a MicroPython runtime, and then reflash the device after inserting the desired Python scripts, or is the process write-only?

Unfortunately the micro:bit could have any other type of program in the nRF51, so we can't count on MicroPython being present.

The reason for the query is that the MicroPython runtime, as built via yotta and distributed with uflash/Mu, is built partly (as far I can see following the yotta build dependency chain) from sources that include non-free code (https://github.com/ARMmbed/ble-nrf51822 for example). Please correct me if this is not the case.

The DAL pulls libraries from Mbed (I believe they are Apache licensed, but haven't got through all of them) and the Nordic nrf51 SDK, which comes with the Nordic soft devices.

The diagram in this link is a bit outdated, but is still a good representation for the MicroPython stack (microbit-dal and microbit-runtime are the interchangeable names): https://lancaster-university.github.io/microbit-docs/concepts/

I think the soft device might be the biggest problem here as it is redistributed as a binary blob without source code. However, the MicroPython build is configured to not add the softdevice (it uses the bbc-microbit-classic-gcc-nosd target). This is because the softdevice contains the BLE functionality and consumes too much memory to run alongside MicroPython, and so all BLE code in the micro:bit DAL has define guards to be compiled-out from MicroPython builds.

@knowledgejunkie I believe all the source code published in those repos have an open source licensed. Was the soft device the only worry?

@jaustin, the question is if it's OK for the firmware for MicroPython on the micro:bit (built, in part, using yotta which is, apparently, non-free) to be distributed in an MIT licensed project like uflash . I.e. as @knowledgejunkie says, we want to check such a situation is compatible with the more restrictive licensing of some of the upstream source (yotta).

Yotta is just a build system, which is Apache licensed. In this case the question is about the source code and blobs retrieved by Yotta to build the DAL and MicroPython.

[ntoll]

@carlosperate awesome response. Thank you. @knowledgejunkie is this enough to answer your query? AFAICT, all the code use in MicroPython for the micro:bit is liberally licensed (@carlosperate, the BBC told me they prefer Apache over MIT, so it makes sense the DAL is Apache).

[jaustin]

Hi all, sorry I'm a bit late to the party....

The DAL is MIT licenced https://github.com/lancaster-university/microbit-dal/blob/master/LICENSE and as Carlos points out, the yotta target we use for building MicroPython (bbc-microbit-classic-gcc-nosd does not include any SoftDevice binaries). https://github.com/lancaster-university/microbit-targets/tree/master/bbc-microbit-classic-gcc-nosd

Aside from that, SoftDevice is licensed by Nordic for redistribution with mbed: https://github.com/ARMmbed/ble-nrf51822/blob/master/LICENSE and https://github.com/ARMmbed/ble-nrf51822/blob/master/softdevice_nrf51822_licence_agreement.txt - I guess it is up to individual projects to rule on the compatibility of that licence with their criteria.

Yotta itself doesn't include the target code, so should be uncomplicated.

[knowledgejunkie]

@carlosperate @jaustin Thank you both very much for your responses and for making explicit the fact that the yotta build target (and final binary) for the micro:bit does not include any SoftDevice binaries.

A key next question is whether it is possible to build the MicroPython runtime with yotta from completely local sources without requiring:

• an mbed account
• a network connection
• SoftDevice binaries present

Having the full source for the MicroPython build chain available and buildable locally is a key requirement for getting projects that include (or strongly depend on) the firmware into Debian main.

(Please note that yotta itself is not yet packaged in Debian.)

[knowledgejunkie]

By way of a short update, I've built the DAL locally on Debian testing/sid (using yotta and uflash from pip whilst their packaging) without the non-dfsg-compatible SoftDevice binaries present, flashed my microbit and tried a few examples, which work \o/.

Building the firmware with the current buildchain dependencies in Debian testing/sid did not work out-of-the-box (as others have also found) but it did seem to generate a working micropython environment. I also noted that the filesize of the generated firmware is different to that recently committed to uflash - I'm thinking this is due to differences in the build toolchain only (maybe @carlosperate or @jaustin can confirm? I am not a cpp developer and installed the deps per the micropython instructions).

In terms of packaging for Debain, the next steps as I see them are:

• package yotta (need to file an ITP, and which is reported to be discontinued upstream...)
• get the firmware building from pristine source in a chroot without network access using the packaged yotta
• split the microbit firmware out into its own package (microbit-firmware? need to file an ITP) as other applications/utilities may want it available
• package uflash, repackaging to remove all copies of the included firmware (firmware.hex and uflash.py) and adding a dependency on the new microbit-firmware package and ensuring it can find the installed default firmware file. The latter issue will require patching to make it DWIM out of the box.

Similarly for mu:

• repackage to remove the included convenience uflash/firmware and add a dependency on the new uflash package and again ensuring it finds the system installed uflash script and firmware file.

With builds working without the SoftDevice binaries present, it looks (to me at least) like the firmware (and any packages which depend on it) is suitable for the main part of the Debian archive \o/

[knowledgejunkie]

Closing this, as building the firmware locally seems possible once yotta and it's missing dependencies are available in Debian.

Thanks for everyone's input.