not for the microbit?

[tcurdt]

I was trying to use this library to do a DFU of the microbit but it appears the services are completely different and hence is not working at all.

https://lancaster-university.github.io/microbit-docs/resources/bluetooth/bluetooth_profile.html

The microbit exposes

uuid_service_dfu_control = CBUUID(string:"E95D93B0-251D-470A-A062-FA1922DFA9A8")
uuid_service_device_info = CBUUID(string:"180A")

What am I missing? Shouldn't this library work with the microbit as well?

[mostafaberg]

I can't confirm that right now, but it seems like the identifier is different, We have a bunch of microbits at work so i'll give one a try and see, but from what it seems jumping to bootloader has a bit of a different mechanism, but my guess is that things should work right after you manage to jump to bootloader, can you try doing that using nRF Connect ? I'll need some time before getting to that task

[tcurdt]

@mostafaberg I assume you mean the DFU from the nRF Toolbox?

With nRF Toolbox I select DFU, the device, the firmware and it's stuck saying "Connecting..."

When I use the nRF Connect I select to connect to the microbit and I don't see nothing more than "Connecting" either.

[mostafaberg]

Yeah that will porbably not work, i actually mean nRF Connect not nRF Toolbox, with nRF Connect you need to connect (and pair depending on your firmware version), then you'll need to manually jump to bootloader, after that the Micro Bit restarts in DFU mode, only then you'll be able to flash.

Does your peripheral require Pairing or does it work without that ?

[tcurdt]

@mostafaberg is does require pairing

you'll need to manually jump to bootloader

Urgh. OK. So IIUC the library only takes care of the second part (after the restart)?

[mostafaberg]

You are pretty much correct, seems like according to their Spec linked in your question that there are two different versions, the old one does not support pairing, and in order to jump to bootloader, you have to write a secret key to characteristic UUIDE95D93B1251D470AA062FA1922DFA9A8

Which is described as:

Writing 0x01 initiates rebooting the micro:bit into the Nordic Semiconductor bootloader if the DFU Flash Code characteristic has been written to with the correct secret key.

Writing 0x02 to this characteristic means "request flash code".

Then there seems to be an updated version that supports pairing (the one you are using currently), which dropped that secret key to be written since you're already bonded to the MicroBit

Then there is an updated version >= 1.7 that states:

DFU Control Service has lost the the DFU Flash Code characteristic since we're now using standard Bluetooth pairing.

So it seems that if you are on firmware 1.7 or newer, you no longer need to use the flash code featre as it's gone, if your MicroBit does not initiate a pairing when you connect to it, that means you have to use the flash code feature to jump to bootloader.

I'm uncertain since I haven't tested this, but according to the spec , and your information provided, to jump to bootloader mode (only after being bonded) do the following:

• Write 0x01 to characteristic UUIDE95D93B1251D470AA062FA1922DFA9A8
• Observe the MicroBit rebooting in bootloader.
• Start the Normal DFU Process by connecting to the MicroBit using this library.

I'm not sure if your bonding info will be persisted after flashing, so on iOS, after flashing the MicroBit you will need to go to Settings->Bluetooth and select your MicroBit in the peripheral list and tap Forget to remove the bonding data from your iOS device.

if bonding persists, you will be able to reconnect, if not you will have issues reconnecting

[tcurdt]

@mostafaberg thanks a lot - that did get me a bit further. After rebooting I call the library and then get this:

Connecting to BBC micro:bit...
centralManager.connect(peripheral, options: nil)
Connecting
[Callback] Central Manager did connect peripheral
Connected to BBC micro:bit
Discovering services...
peripheral.discoverServices(nil)
2017-08-24 12:29:16.712745+0200 Calliope[10056:4409703] [CoreBluetooth] WARNING: The delegate for <CBPeripheral: 0x1740f5180, identifier = 19D1B789-0D1A-4103-8E2E-80C981934FF1, name = BBC micro:bit, state = connected> does not implement -[peripheral:didModifyServices:]
Services discovered
Starting Legacy DFU...
Connected to BBC micro:bit
Services discovered
Legacy DFU Service found
Discovering characteristics in DFU Service...
peripheral.discoverCharacteristics(nil, for: 00001530-1212-EFDE-1523-785FEABCD123)
DFU characteristics discovered
Reading DFU Version number...
peripheral.readValue(00001534-1212-EFDE-1523-785FEABCD123)
Read Response received from 00001534-1212-EFDE-1523-785FEABCD123, value (0x): 0400
Version number read: 0.4
Writing to characteristic 00001531-1212-EFDE-1523-785FEABCD123...
peripheral.writeValue(0x06, for: 00001531-1212-EFDE-1523-785FEABCD123, type: .withResponse)
[Callback] Central Manager did disconnect peripheral
Disconnected by the remote device
The init packet is required by the target device

As you can see - unfortunately the upload does not go through. Seems like the device disconnect - for some reason. Any thoughts?

[mostafaberg]

That's perfect, seems like you got it working already, your issue is in the last line in the log

The init packet is required by the target device

You need to have an init packet in the DFU file, here is how to create one

After fixing that, i'm sure your firmware will work flawlessly :)

[tcurdt]

@mostafaberg sounds good - progress :)

IIUC I need to create a zip and add a file with crc information to that zip? ATM I only have a hex - no zip and I was under the impression the library would generate the required init packet.

I am on macOS not windows. So those .exe don't help much - but I got hex2bin compiled. Is the crc a crc32?

[tcurdt]

Als: is there a DFU file out there where the init package is available? So I can verify my init package generation?

[mostafaberg]

Hmm good question, @philips77 do you know where to find ready made ones ?

[philips77]

Hi, afaik there are in SDK in dfu folder. You'll find there hex, but also test zip files. Also here: https://github.com/NordicSemiconductor/Android-nRF-Connect/tree/master/documentation/Automated%20tests/DFU%20Tests

To create init file you should use nrfutil python app, read more here https://github.com/NordicSemiconductor/pc-nrfutil. The pdf given above is outdated, i have to update it.

[philips77]

For legacy dfu, use version 0.5.2 of nrfutil, the newest one isn't compatible.

[tcurdt]

What I still don't get is:

Via USB I can upload hex file just fine. No further metadata needed. Does it just assume the metadata and generates the checksums on the fly?

From the API it looks like the library does support hex, bin and zip. Now when I try the hex way the init packet is missing.

If it's required, wouldn't it make sense to be able to specify the required information as parameter to the DFUFirmware or DFUStream objects?

[tcurdt]

Is the pem a requirement? BTW: Seems we are using SDK version 10.0.0

[philips77]

Nope, pem is a requirement in Secure DFU. In legacy it was added in SDK 8(?) and optional/hard to enable.

[philips77]

Init pack is required for dfu since sdk 7.1. It contains fw and hw version and CRC which a bit prevents from flashing wrong fw on a different device. It's not secure, as the init packet isn't signed, and you may remove the app without actually having any zip packet. You may just send that you are about to send new SD, which will clear app to make space for new sd in dual bank. But protects from stupid mistakes.

It's up to the bootloader to require this or that. In the lib I can't say what's needed before starting dfu.

[tcurdt]

@philips77 thanks for helping in clearing things up but...

if the init packet is not signed - what's the pem for? And when I look at the protobuf at https://github.com/NordicSemiconductor/pc-nrfutil/blob/master/nordicsemi/dfu/dfu-cc.proto I see a SignedCommand which holds a signature.

So what I did now is generate some keys (I wouldn't know where to find the proper key) and then the zip

nrfutil keys generate private.pem
nrfutil pkg generate --hw-version 51 --sd-req 0x80 --application-version 4 --application app.hex --key-file private.pem app_dfu_package.zip

When I try to upload the zip (via DFUFirmware(urlToZipFile: url)) I get

uploading file:///var/containers/Bundle/Application/22CB2315-C0E0-4283-A3E5-73A7B0892069/Calliope.app/app_dfu_package.zip
Connecting to BBC micro:bit [vatuz]...
centralManager.connect(peripheral, options: nil)
Connecting
[Callback] Central Manager did connect peripheral
Connected to BBC micro:bit [vatuz]
Discovering services...
peripheral.discoverServices(nil)
2017-08-25 13:30:38.956825+0200 Calliope[10687:4625737] [CoreBluetooth] WARNING: The delegate for <CBPeripheral: 0x1740e3e80, identifier = 19D1B789-0D1A-4103-8E2E-80C981934FF1, name = BBC micro:bit [vatuz], state = connected> does not implement -[peripheral:didModifyServices:]
Services discovered
Starting Legacy DFU...
Connected to BBC micro:bit [vatuz]
Services discovered
Legacy DFU Service found
Discovering characteristics in DFU Service...
peripheral.discoverCharacteristics(nil, for: 00001530-1212-EFDE-1523-785FEABCD123)
DFU characteristics discovered
Reading DFU Version number...
peripheral.readValue(00001534-1212-EFDE-1523-785FEABCD123)
Read Response received from 00001534-1212-EFDE-1523-785FEABCD123, value (0x): 0400
Version number read: 0.4
Enabling notifications for 00001531-1212-EFDE-1523-785FEABCD123...
peripheral.setNotifyValue(true, for: 00001531-1212-EFDE-1523-785FEABCD123)
Starting
Notifications enabled for 00001531-1212-EFDE-1523-785FEABCD123
DFU Control Point notifications enabled
Writing to characteristic 00001531-1212-EFDE-1523-785FEABCD123...
peripheral.writeValue(0x0104, for: 00001531-1212-EFDE-1523-785FEABCD123, type: .withResponse)
Writing image sizes (0b, 0b, 257056b) to characteristic 00001532-1212-EFDE-1523-785FEABCD123...
peripheral.writeValue(0x000000000000000020ec0300, for: 00001532-1212-EFDE-1523-785FEABCD123, type: .withoutResponse)
Data written to 00001531-1212-EFDE-1523-785FEABCD123
Start DFU (Op Code = 1, Upload Mode = 4) request sent
Notification received from 00001531-1212-EFDE-1523-785FEABCD123, value (0x): 100104
Response (Op Code = 1, Status = 4) received
Error 4: Data size exceeds limit
Writing to characteristic 00001531-1212-EFDE-1523-785FEABCD123...
peripheral.writeValue(0x06, for: 00001531-1212-EFDE-1523-785FEABCD123, type: .withResponse)
[Callback] Central Manager did disconnect peripheral
Disconnected by the remote device
Data size exceeds limit

Which is a bit odd. The hex file uploads and works just fine via USB.

I am wondering if it tries to cram into the application space what really is softdevice + application. Could that be possible? Is there a way to check? I didn't create that file.

[philips77]

So.. as I wrote, you have to use nrfutil v0.5.2, not the 3.2.0. You generate the init file for Secure DFU (SDK 12+), but as you say you are using SDK 10 (legacy DFU). There is no pem, no proto file, etc in legacy DFU.

[philips77]

Also, from the log I see you changed the number returned from DFU Version char to 0x04:

Reading DFU Version number...
peripheral.readValue(00001534-1212-EFDE-1523-785FEABCD123)
Read Response received from 00001534-1212-EFDE-1523-785FEABCD123, value (0x): 0400
Version number read: 0.4

May I ask why? This should not be changed, as based on this number the lib determines features of your botloader and know how to handle it. In SDK 10 (since SDK 8) the value returned should be 0600. Check here: https://github.com/NordicSemiconductor/IOS-Pods-DFU-Library/blob/2969438633ef1f242e3ab8621ea7f13a54b44823/iOSDFULibrary/Classes/Implementation/DFUServiceInitiator.swift#L108

[philips77]

To use nrfutil 0.5.2 (actually I see now 0.5.3 is latest) you have to clone the repo and change branch to 0.5.3: https://github.com/NordicSemiconductor/pc-nrfutil/tree/0_5_3. You can't install it together with the newest one as they have the same package names. First uninstall 3.2.0 using pip.

[tcurdt]

Also, from the log I see you changed the number returned from DFU Version char to 0x04: May I ask why? This should not be changed, as based on this number the lib determines features of your bootloader and know how to handle it. In SDK 10 (since SDK 8) the value returned should be 0700 or 0800, as far as I remember. Check here:

I am just the poor guy trying to get the DFU working from iOS :) I don't know why they changed the version. I will pass that on. I didn't find the version specific code paths in the library yet.

Sorry for the 3.2.0 - totally missed that. Now I installed 0.5.2 (can also upgrade to 0.5.3)

nrfutil dfu genpkg \
  --application app.hex \
  app_dfu_package.zip

but I am still getting Data size exceeds limit

[philips77]

nrfutil dfu genpkg --application app.hex --application-version 0xffffffff --dev-type 0xffff --dev-revision 0xffffffff --sd-req 0x64 app_dfu_package.zip

The DFU packet (dat file) should be (8+4+4+4+4+4) 28 bytes long. The app.bin inside should be small enough to fit between the old app and bootloader. Could you put the new log here?

0x64 - is the ID of the SD 8.0.0.

[tcurdt]

I also tried

nrfutil dfu genpkg \
  --softdevice app.hex \
  app_dfu_package.zip

I do see a difference:

Writing image sizes (0b, 0b, 257056b) to characteristic 00001532-1212-EFDE-1523-785FEABCD123...
peripheral.writeValue(0x000000000000000020ec0300, for: 00001532-1212-EFDE-1523-785FEABCD123, type: .withoutResponse)
Data written to 00001531-1212-EFDE-1523-785FEABCD123
Start DFU (Op Code = 1, Upload Mode = 4) request sent

vs

Writing image sizes (257056b, 0b, 0b) to characteristic 00001532-1212-EFDE-1523-785FEABCD123...
peripheral.writeValue(0x20ec03000000000000000000, for: 00001532-1212-EFDE-1523-785FEABCD123, type: .withoutResponse)
Data written to 00001531-1212-EFDE-1523-785FEABCD123
Start DFU (Op Code = 1, Upload Mode = 1) request sent

but it still fails with the data size exceeded.

[tcurdt]

@philips77 I tried this

nrfutil dfu genpkg \
  --application app.hex \
  --application-version 0xffffffff \
  --sd-req 0x64 \
  --dev-type 0xffff \
  app_dfu_package.zip
  # --dev-revision 0xffffffff \

The value for dev revision seems to be a problem and I get a stacktrace. Without it I am getting the following zip:

  257056  Stored   257056   0% 08-25-2017 14:46 9d0bdfc3  app.bin
      14  Stored       14   0% 08-25-2017 14:46 f09623f9  app.dat
     492  Stored      492   0% 08-25-2017 14:46 1e4c281f  manifest.json

but that shows 14 - not 28 bytes for the dat file. But looking at this: Given that the device has 256kb flash no wonder it doesn't fit I guess.

The log is still the same:

Writing to characteristic 00001531-1212-EFDE-1523-785FEABCD123...
peripheral.writeValue(0x0104, for: 00001531-1212-EFDE-1523-785FEABCD123, type: .withResponse)
Writing image sizes (0b, 0b, 257056b) to characteristic 00001532-1212-EFDE-1523-785FEABCD123...
peripheral.writeValue(0x000000000000000020ec0300, for: 00001532-1212-EFDE-1523-785FEABCD123, type: .withoutResponse)
Data written to 00001531-1212-EFDE-1523-785FEABCD123
Start DFU (Op Code = 1, Upload Mode = 4) request sent
Notification received from 00001531-1212-EFDE-1523-785FEABCD123, value (0x): 100104
Response (Op Code = 1, Status = 4) received
Error 4: Data size exceeds limit
Writing to characteristic 00001531-1212-EFDE-1523-785FEABCD123...
peripheral.writeValue(0x06, for: 00001531-1212-EFDE-1523-785FEABCD123, type: .withResponse)
[Callback] Central Manager did disconnect peripheral
Disconnected by the remote device
Data size exceeds limit

[philips77]

14 is OK, I was counting hex as 2 ;) Sorry!

[philips77]

Well.. if your app is 257056 bytes then it may be too large to do a DFU with dual bank. Then the flash memory contains MBR (0-0x1000), SD (0x1001-?), old app (?), bootloader (?-?). The space between old app and bl is used to store the new app (second bank, in case the upload failed you still have the old one and device is working). Try compiling your app with space optimization or... reduce functionality. I guess you can't add more flash to the hw. You may also change to single bank update, but with legacy DFU it may require flashing new bootloader as they support either dual or single bank). In secure DFU single bank is automatically on when not enough space for dual.

[philips77]

To check what's the maximum available size you may try to cut some bytes from your "zip" and send smaller file until it works. Then tell your fw colleges that that's the limit.

[tcurdt]

OK. So I just checked:

The hex that I was trying to flash as application is either BL+SD+APP or SD+APP - hence the size (problems). It's basically the hex used for flashing via USB. Not sure what that is supposed to look like.

I guess if I'd know the position of the various parts I could split the file and could then create the proper zip that should work for DFU via BLE.

[philips77]

Hi, then you are lucky. The SD can't be uploaded together with an App. You have to have 2 HEX files. If you can't get them from your fw team, you may split them manually (however it's not that trivial). If you want to upload SD(+BL) as well, you must have 2(3) hex files and then pass them to nrfutil as --application app.hex --softdevice sd.hex --bootloader.hex. It will automatically merge the SD+BL, but the app will be as a separate file, sent in the second connection. If you just update the app - that's easier, just give the --application param and it will work.

[tcurdt]

I finally got it working. The original hex file was for upload via USB. I had to manually cut out the application part. Then after creating the dat file the upload worked and so does the app. This really needs some more documentation - but I think it's safe to close the issue for now.