Firmware update over Bluetooth LE

[endian-albin]

Follow lupyuen's plan when implementing this crucial feature.

Current status:

• [x] bootloader support
• [x] support for writing to SPI flash
• [x] allow fetching firmware images over BLE
• [x] refresh watchdog if present
• [x] build DFU images with Github CI

[endian-albin]

From the PineTime chat on August 8, 2020:

[D] <@lupyuen> may need to run adafruit-nrfutil to package it for DFU [D] <@lupyuen> yeah so the problem is that JF's firmware accepts only SoftDevice DFU packages [D] <@lupyuen> so the firmware needs to be packaged as a DFU zip file [D] <@lupyuen> lemme show you the steps... [D] <@lupyuen> https://lupyuen.github.io/pinetime-rust-mynewt/articles/cloud#make-dfu-firmware-pinetime-mcuboot-app [D] <@lupyuen> this is explained in my doc here [D] <@lupyuen> we need to run adafruit-nrfutil to create the zip package [D] <@lupyuen> the zip package will contain 3 files: the bin firmware, a manifest and a dat file [D] <@lupyuen> the manifest will tell nRF Connect to flash the bin at address 0x8000 [D] <@lupyuen> hope this makes sense? endian-albin: I haven’t implemented bootloader support in the Hypnos firmware though. [D] <@lupyuen> cant we link mcuboot [header] into the firmware? [D] <@lupyuen> lemme whether this is documented.... [D] <@lupyuen> https://juullabs-oss.github.io/mcuboot/readme-zephyr.html [D] <@lupyuen> i think thats the doc endian-albin: Yeah I suppose. So, should I focus on creating a DFU zip file that can be flashed from InfiniTime then? [D] <@lupyuen> oh we need to get the mcuboot header in [D] <@lupyuen> then create the DFU package [D] <@lupyuen> hmmm how do we get the mcuboot header in... [D] <@lupyuen> i use imgtool.py for mynewt. also for infinitime [D] <@lupyuen> would that work for zephyr too? [D] <@lupyuen> oh the link above mentions imgtool.py too [D] <@lupyuen> so i think it should work [D] <@lupyuen> so remember that the firmware should start at 0x8000 [D] <@lupyuen> the first 32 bytes are for the mcuboot header, followed by the vector table: https://lupyuen.github.io/pinetime-rust-mynewt/articles/dfu#mcuboot-bootloader-for-pinetime [D] <@lupyuen> the mcuboot library will be linked into the firmware to set the OK status of the firmware [D] <@lupyuen> boot_set_confirmed() [D] <@lupyuen> if we dont call this function, mcuboot will rollback at the next reboot

[sdorre]

I would go for the Zephyr way. Using MCUBoot as bootloader and DFU. https://docs.zephyrproject.org/latest/guides/device_mgmt/dfu.html#dfu

There is an example : https://github.com/zephyrproject-rtos/zephyr/tree/master/samples/subsys/mgmt/mcumgr/smp_svr

[endian-albin]

The PineTime will soon ship with a firmware with a slightly different update mechanism and if we don't support it, Hypnos will be less relevant as it can't be easily installed.

[lupyuen]

Hi All: Would be great to have pinetime-hypnos flashing to PineTime over Bluetooth! Lemme explain the complicated situation now...

1. Pine64 is now preparing to preload the next batch of PineTime watches with Community Firmware: The port of FreeRTOS by JF, named InfiniTime

2. Also preloaded is the MCUBoot Bootloader, which I have added some minor extensions based on Mynewt.

3. When PineTime Owners receive their new watches, they will be able to update / replace the firmware wirelessly with the nRF Connect mobile app (Android and iOS)

4. PineTime Owners can still flash their watches the wired way via SWD. The new watches will no longer have Flash ROM Protection. I have been testing Pogo Pins with PineTime, and Pine64 is thinking of selling them.

5. Based on market demand, Pine64 is also thinking of selling Sealed PineTime Watches as an option to customers. These watches will not have SWD port access. Yes there's a possibility that the watches will get bricked, so we have to make it very clear that our customers will only flash production, non-experimental firmware. It's like Long Term Support (LTS).

I see a huge opportunity for pinetime-hypnos because...

1. InfiniTime (FreeRTOS) is currently the only firmware that may be flashed wirelessly to the upcoming PineTime watches

2. wasp-os (MicroPython) and ATCWatch (Arduino) are incompatible with the wireless flashing mechanism. They run on Nordic SoftDevice and we couldn't reconcile the SoftDevice stack with the NimBLE stack used by InfiniTime. So for Sealed PineTime Watches, they can't be flashed with wasp-os and ATCWatch.

3. Mynewt (which I'm working on) can technically be flashed since it runs on NimBLE too. But I decided to put Mynewt on hold while I help JF with the InfiniTime launch.

So pinetime-hypnos could be the alternative OS that will be flashed to PineTime watches. And for Sealed PineTime Watches, we may have only 2 possible options: pinetime-hypnos and InfiniTime.

This is getting long, so I'll write the "How To" in the next post :-)

[lupyuen]

How do we package pinetime-hypnos for flashing to PineTime?

1. The overall approach for packaging pinetime-hypnos for MCUBoot is described here...

   "Building and using MCUboot with Zephyr"

   The article refers to this sample app...

   mcuboot/samples/zephyr/hello-world

2. The firmware needs to start at address 0x8020 instead of 0x0.

   0x8000 to 0x801F will contain the 32-byte MCUBoot Image Header (I'll explain later)

   0x8020 onwards will contain the Interrupt Vector Table followed by the firmware code

   Check out the diagram here...

   "MCUBoot Bootloader for PineTime"

3. According to the Zephyr docs (I may be wrong), we need to set CONFIG_BOOTLOADER_MCUBOOT=y

   CONFIG_BOOTLOADER_MCUBOOT Doc

   We may need to set CONFIG_ROM_START_OFFSET to 0x8000 or 0x8020

   CONFIG_ROM_START_OFFSET Doc

4. To insert the 32-byte MCUBoot Image Header, we'll call imgtool.py

   imgtool Doc

   I wrote about imgtool.py here...

   "Generate a Firmware Image File for PineTime"

   For reference, here are the imgtool.py commands used for the InfiniTime build script...

   #  Install MCUBoot
   git clone --branch v1.5.0 https://github.com/JuulLabs-OSS/mcuboot
   #  Install imgtool dependencies
   pip3 install --user -r mcuboot/scripts/requirements.txt    
   #  Create firmware image
   mcuboot/scripts/imgtool.py create --align 4 --version 1.0.0 --header-size 32 --slot-size 475136 --pad-header pinetime-mcuboot-app.bin pinetime-mcuboot-app-img.bin
   mcuboot/scripts/imgtool.py verify pinetime-mcuboot-app-img.bin

   where pinetime-mcuboot-app.bin is the BIN firmware created by the FreeRTOS build.

   This creates pinetime-mcuboot-app-img.bin, the firmware image that contains the MCUBoot Image Header.

5. This part is unusual: InfiniTime uses the SoftDevice DFU Protocol for flashing, even though it doesn't use SoftDevice.

   This means InfiniTime emulates the SoftDevice DFU Protocol, so DFU packages need to follow the SoftDevice DFU format.

   Someday we may switch to the MCU Manager Simple Management Protocol. I'm working with JF on this.

6. Note that the nRF Connect mobile app (Android and iOS) supports both SoftDevice Protocol and Simple Management Protocol for DFU.

   But the file format is different: SoftDevice DFU packages are ZIP files, Simple Management Protocol DFU packages are BIN files (which is the output from imgtool.py)

7. To create a SoftDevice DFU Zip Package from the imgtool.py output, we call adafruit-nrfutil...

    #  Install adafruit-nrfutil
    pip3 install --user wheel
    pip3 install --user setuptools
    pip3 install --user adafruit-nrfutil
   #  Create DFU package
    adafruit-nrfutil dfu genpkg --dev-type 0x0052 --application pinetime-mcuboot-app-img.bin pinetime-mcuboot-app-dfu.zip    

   where pinetime-mcuboot-app-img.bin is the output from imgtool.py

   Note that adafruit-nrfutil must be passed a file with extension .bin. Otherwise it assumes the file is in .hex format and fails with a UTF decoding error.

   This creates a SoftDevice DFU Package pinetime-mcuboot-app-dfu.zip that may be used with nRF Connect to flash PineTime wirelessly.

   The above script comes from the GitHub Actions Workflow for InfiniTime...

   pinetime-lab/.github/workflows/main.yml

   The script is explained here...

   "Create DFU Package"

This sounds complicated but I'm really keen to help make pinetime-hypnos flashable to the new batch of PineTime watches... Because it's always good to offer firmware options to PineTime Owners. Thanks :-)

[sdorre]

But What about the Memory layout ? I guess we should adopt somehow the same layout right ? Is there any plan on putting the FW on the flash memory ? Is that all dual bank based ? So that if the new FW does not start, the device is still starting ?

Apparently the FW images are signed. Which means, if we want to flash the hypnos or InitiTime, we have to share the encryption keys. (I guess it is not so much of a security issue, if we want the device to remain open) But we need to keep that in mind.

[MirkoCovizzi]

2\. wasp-os (MicroPython) and ATCWatch (Arduino) are incompatible with the wireless flashing mechanism. They run on Nordic SoftDevice and we couldn't reconcile the SoftDevice stack with the NimBLE stack used by InfiniTime. So for Sealed PineTime Watches, they can't be flashed with wasp-os and ATCWatch.

@lupyuen First of all, I'm not finding it fair to the community that is a possibility. At minimum the bootloader present in the PineTime (both sealed and unsealed) should at least make it possible to flash a different bootloader, for example the one based on Adafruit's. The person using the watch should have a saying in the choice of firmware and bootloader they want to run and enforcing a particular type of bootloader (like MCUBoot) seems to be opposite to the concept of open hardware and open software.

[JF002]

But What about the Memory layout ? I guess we should adopt somehow the same layout right ? Is there any plan on putting the FW on the flash memory ? Is that all dual bank based ? So that if the new FW does not start, the device is still starting ?

The memory layout is described here. It is quite straightforward. The bootloader, the running firmware and the scratch area are in the internal flash memory, the bootloader graphics and the secondary area are in the external SPI flash. The effectively allows MCUBoot to rollback to the previous version of the firmware in case of issue.

Apparently the FW images are signed. Which means, if we want to flash the hypnos or InitiTime, we have to share the encryption keys. (I guess it is not so much of a security issue, if we want the device to remain open) But we need to keep that in mind.

We do not sign the images for now.

First of all, I'm not finding it fair to the community that is a possibility. At minimum the bootloader present in the PineTime (both sealed and unsealed) should at least make it possible to flash a different bootloader, for example the one based on Adafruit's. The person using the watch should have a saying in the choice of firmware and bootloader they want to run and enforcing a particular type of bootloader (like MCUBoot) seems to be opposite to the concept of open hardware and open software.

I totally agree with you! In a perfect world, the user should be able to choose the bootloader and the firmware he wants to use. And we've talked about this many times! But there are technical issues that are not easy to solve. I won't go into details here, but the NRF softdevice and bootloader are close-source and the way they work makes it nearly impossible to replace them with another bootloader and ble stack. Well, it may be possible, but we haven't found a solution yet. If anyone has an idea about that, let's talk!

The MCUBoot based bootloader is more open, in the way that it does not make any assumptions on the BLE stack you have to use in your application firmware. You can use the one you want (except NRF Softdevice...).

[endian-albin]

I suggest that discussions involving the PineTime community as a whole take place in the PineTime chat. Please come there; the people are very friendly :)

[endian-albin]

I'll start working on this by making Hypnos installable from MCUBoot+InfiniTime, even if it can't initially repay the favour. I'll try automatically produce DFU zip files using Github Actions. Then, from inside Hypnos, we could make new firmware images installable using either the SoftDevice Protocol or the Simple Management Protocol.

[endian-albin]

@lupyuen, considering all that that's been said thus far, how would you modify the part in the device tree source below that deals with partitions? Since we need to use the external flash, I suppose we'll need to move slot1 to a new &flash1 node...

&flash0 {
    /*
     * For more information, see:
     * http://docs.zephyrproject.org/latest/guides/dts/index.html#flash-partitions
     */
    partitions {
        compatible = "fixed-partitions";
        #address-cells = <1>;
        #size-cells = <1>;

        boot_partition: partition@0 {
            label = "mcuboot";
            reg = <0x00000000 0xc000>;
        };
        slot0_partition: partition@c000 {
            label = "image-0";
            reg = <0x0000C000 0x32000>;
        };
        slot1_partition: partition@3e000 {
            label = "image-1";
            reg = <0x0003E000 0x32000>;
        };
        scratch_partition: partition@70000 {
            label = "image-scratch";
            reg = <0x00070000 0xa000>;
        };
        storage_partition: partition@7a000 {
            label = "storage";
            reg = <0x0007a000 0x00006000>;
        };
    };
};

[endian-albin]

Damn, just adding the lines below to prj.conf generates an image that is too big for internal memory even with the debug log disabled (108 %) .

CONFIG_FLASH=y
CONFIG_BOOTLOADER_MCUBOOT=y
CONFIG_MCUBOOT_IMG_MANAGER=y

So the secondary area really must be in the external flash :)

[JF002]

So the secondary area really must be in the external flash :)

@lupyuen MCUBoot bootloader expects that the secondary slot is in the external flash anyway :) I don't know Zephyr, but are you sure you need to add CONFIG_BOOTLOADER_MCUBOOT? The bootloader is built in another binary, so you don't need it in your application.

I'll start working on this by making Hypnos installable from MCUBoot+InfiniTime

I think the only thing you need to do is change the linker script to make your application start at offset 0x8000, set the valid bit in flash and refresh the watchdog (to be future proof). You can have a look at my "FirmwareValidator" to see how to manually set the valid flag : https://github.com/JF002/Pinetime/blob/manualValidation/src/Components/FirmwareValidator/FirmwareValidator.cpp.

Then, of course, you'll need to implement the NRF DFU Protocol or the Simple Management Protocol to provide OTA functionality.

[lupyuen]

Yep JF is correct. You can compare the Zephyr Flash Map with the Mynewt one used by MCUBoot...

https://github.com/lupyuen/pinetime-rust-mynewt/blob/master/hw/bsp/nrf52/bsp.yml

Device 0 is Internal Flash ROM, Device 1 is External SPI Flash

[endian-albin]

So the secondary area really must be in the external flash :)

MCUBoot bootloader expects that the secondary slot is in the external flash anyway :) I don't know Zephyr, but are you sure you need to add CONFIG_BOOTLOADER_MCUBOOT? The bootloader is built in another binary, so you don't need it in your application.

According the Zephyr documentation, the "application’s .conf file needs to enable the CONFIG_BOOTLOADER_MCUBOOT Kconfig option in order for Zephyr to be built in an MCUboot-compatible manner". I'm not sure how much of the documentation will still be relevant since we'll not exactly be doing things in a Zephyr-idiomatic way here, but I don't think this config by itself is to blame for the size problem regarding SRAM. What it seems to be doing though, among other things, is to enable the partition layout in the device tree source which currently declares that both slot 0 and 1 reside in the internal flash and then SRAM will not be enough.

I'll start working on this by making Hypnos installable from MCUBoot+InfiniTime

I think the only thing you need to do is change the linker script to make your application start at offset 0x8000,

You mean the linker script that you already have for InfiniTime and Mynewt+Rust?

set the valid bit in flash and refresh the watchdog (to be future proof). You can have a look at my "FirmwareValidator" to see how to manually set the valid flag : https://github.com/JF002/Pinetime/blob/manualValidation/src/Components/FirmwareValidator/FirmwareValidator.cpp.

Right, so I'll simply write a raw value to a specific address and won't need any special Zephyr functions for talking with the boot loader? If you think we can pretend that MCUBoot doesn't exist from the point of view of the application, our work will be much easier. @lupyuen do you also think so? It's not exactly what you seem to have suggested above.

Then, of course, you'll need to implement the NRF DFU Protocol or the Simple Management Protocol to provide OTA functionality.

We'll need to figure out how to write to the external flash first. Would it then be enough write bytes continuously to slot 1, starting from a certain offset, manually set a bootloader bit and reboot or would we need to take into account the entire partition layout of the external flash?

[JF002]

Right, so I'll simply write a raw value to a specific address and won't need any special Zephyr functions for talking with the boot loader?

Yes ;) In fact, we designed the bootloader so that it is as little intrusive as possible for the application firmware. For now, the low-level requirements are

• start offset : 0x8000
• write the valid bit at a specific address in internal flash memory (I think MCU Manager Simple Management Protocol implements this, but for now I've decided to simply hardcode the adress and the value).
• refresh the watchdog

Appart from these, the application firmware does not need to know it's running with mcuboot. This is why I can easily build 2 versions of my firmware (one standalone and one for mcuboot) : the very same code is compiled and linked against 2 linker scripts (https://github.com/JF002/Pinetime/blob/master/gcc_nrf52.ld and https://github.com/JF002/Pinetime/blob/master/gcc_nrf52-mcuboot.ld)

This is the same for the OTA : as long as your firmware writes the new image at the correct position in the external flash, you don't have to know anything about mcuboot to implement it.

But I don't know how to translate these requirements in Zephyr.

We'll need to figure out how to write to the external flash first. Would it then be enough write bytes continuously to slot 1, starting from a certain offset, manually set a bootloader bit and reboot or would we need to take into account the entire partition layout of the external flash?

On the low-level, yes, all you need to do is writing all the bytes continuously starting from a certain offset (do not forget to erase the flash pages before write). That's exactly what I'm doing here : https://github.com/JF002/Pinetime/blob/master/src/Components/Ble/DfuService.cpp#L160 Note that the valid bit must be written in the internal memory, after mcuboot swaps the old firmware with the new one.

[endian-albin]

I've successfully tested NOR flash erase, write and read using the spi_flash sample.

JEDEC SPI-NOR SPI flash testing
==========================

Test 1: Flash erase
Flash erase succeeded!

Test 2: Flash write
Attempting to write 4 bytes
Data read matches data written. Good!!

It required the following additional settings:

CONFIG_FLASH=y
CONFIG_SPI_NOR=y
CONFIG_LEGACY_DEVICETREE_MACROS=y

[endian-albin]

I'm now trying to do things closer to the "Zephyr way" by describing the partition layout in the devicetree source and including the standard functions for reading and setting the boot confirm bit.

[endian-albin]

Now linking doesn't work when including functions such as boot_write_img_confirmed(): undefined reference to boot_is_img_confirmed

Fixed by more config magic: CONFIG_IMG_MANAGER=y

[endian-albin]

Things are progressing. I'm now at step 4: "To insert the 32-byte MCUBoot Image Header". Maybe that's even handled by Zephyr. If so, I'm at step 7.

[endian-albin]

@lupyuen has explained that the interrupt vector table is in the wrong location. His proposed workaround is to copy a number of bytes from address 8200 to 8020 by modifying the zephyr.bin file directly. I don't know of any other solution right now (we tried a custom linker script and failed), but if we do it this way I hope the function for confirming the image will still work. Another potential problem relates to the hex and elf images -- they will no longer correspond with the modified bin file used in the dfu package.

[sdorre]

Why don"t we simply follow the Zephyr setup first ? They made it work isn't it ? and then let's see what differs from that ?

This issue with the IVT happen with Hypnos as well ? is only in InfiniTime ?

[endian-albin]

Why don"t we simply follow the Zephyr setup first ? They made it work isn't it ? and then let's see what differs from that ?

We could try that, but we would not be able to boot from (the next) factory image, because of the bootloader's expectations of memory layout.

This issue with the IVT happen with Hypnos as well ? is only in InfiniTime ?

It happens with Hypnos but not with InfiniTime when they use their custom linker script.

[lupyuen]

I have patched the firmware file, by copying the Vector Table to the right address....

https://github.com/lupyuen/patch-bin/blob/master/patch-bin.c

The original and patched files are here...

https://github.com/lupyuen/patch-bin/releases/tag/v1.0.0

The patched firmware won't start on my PineTime, so now I'm debugging with gdb. But debugging the firmware gets complicated because there's a bootloader at 0x0, which isn't part of the ELF file. I'm now creating a Stub Bootloader that contains only the Hypnos Vector Table. Please hang on...

BTW I'm using ST-Link, not JLink, hope that won't be a problem for the firmware. Thanks!

[endian-albin]

Many thanks for helping Hypnos become relevant!

BTW I'm using ST-Link, not JLink, hope that won't be a problem for the firmware. Thanks!

Not at all! Being dependent on JLink is a serious bug IMHO.

[endian-albin]

I just found this upstream commit related vector table alignment. It's just one month old, more recent than v2.3 that we're currently on.

[lupyuen]

Thanks :-) Some updates...

The patch-bin utility now writes the vector table into another file zephyr-vector.bin...

https://github.com/lupyuen/patch-bin/blob/master/patch-bin.c

zephyr-vector.bin will work as a simple bootloader, since it contains only the vector table. I have loaded zephyr-vector.bin at address 0x0 and the debugger is running fine with the ELF file from pinetime-hypnos-fw-f61aaf1

The original and updated patched files and vector table file are here...

https://github.com/lupyuen/patch-bin/releases/tag/v1.0.0

I have traced the firmware problem to some Arm Fault. I'm now stepping through the source code to see where this fault was triggered. The VSCode debugger config is here: (uses the VSCode Cortex-Debug Extension)

https://github.com/lupyuen/pinetime-hypnos/blob/master/.vscode/launch.json

https://github.com/lupyuen/pinetime-hypnos/blob/master/workspace.code-workspace

[lupyuen]

The firmware seems to be initialising the logger and never returns. Can you send me another build with logging disabled? The firmware could be hanging because I'm using ST-Link, not JLink. Thanks! Here's the VSCode Debugging video: https://youtu.be/964wf9U1dvM

[lupyuen]

The files have been updated from the logging disabled build hypnos-no-logging...

https://github.com/lupyuen/patch-bin/releases/tag/v1.0.0

With the logging disabled build hypnos-no-logging/zephyr.elf, it seems to be hanging inside gpio_nrfx_init.

UPDATE: Nope it's not hanging there. The debugger was confused. Maybe we need to remove the optimise options "-O" from gcc?

Lemme investigate...

[lupyuen]

Wonder if you could help me check...

If you flash this to 0x0... https://github.com/lupyuen/patch-bin/releases/download/v1.0.0/zephyr-vector.bin

And this to 0x8000... https://github.com/lupyuen/patch-bin/releases/download/v1.0.0/zephyr-img.bin

Do you see anything?

[endian-albin]

Wonder if you could help me check...

If you flash this to 0x0...

https://github.com/lupyuen/patch-bin/releases/download/v1.0.0/zephyr-vector.bin

And this to 0x8000...

https://github.com/lupyuen/patch-bin/releases/download/v1.0.0/zephyr-img.bin

Do you see anything?

I can check in about 12 hours, but could it be an issue with the firmware starting at 0x8020 and not 0x8000?

[endian-albin]

@lupyuen: I see nothing I'm afraid. I've also tried to debug the latest master (we just merged a new touch sensor driver). With no optimizations and setting the break point at main, GDB gave me this:

Program received signal SIGTRAP, Trace/breakpoint trap.
0x0002e138 in bt_gatt_attr_read_service (conn=0x250ff <sys_slist_get_not_empty+38>, attr=0xc6c5 <ticker_job+1828>, buf=0x20008d00 <z_interrupt_stacks+944>, len=0, offset=0)
    at /home/albin/projekt/pt/pinetime-hypnos/zephyr/subsys/bluetooth/host/gatt.c:1172
1172            return bt_gatt_attr_read(conn, attr, buf, len, offset,

Maybe we should now try to do what @sdorre suggested: boot Hyponos from MCUBoot configured the Zephyr way. Thus far we've only tried running the firmware without a bootloader.

[lupyuen]

Hang on... I think I found out what's wrong...

The SCB VTOR Register in Arm Cortex-M points the Vector Table...

http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.dui0552a/BABIFJFG.html

For Arm Cortex-M the VTOR register is located at 0xE000ED08, so we can dump out the register with gdb like this...

x/x 0xE000ED08
0xe000ed08: 0x00008200

VTOR Register says that the Vector Table is at 0x8200. But what's at 0x8200?

x/80x 0x8200
0x8200: 0x00000000  0x00000000  0x00000000  0x00000000
0x8210: 0x00000000  0x00000000  0x00000000  0x00000000
0x8220 <_vector_table>: 0x20008d00  0x0000c6c5  0x000250ff  0x0000c6f5
0x8230 <_vector_table+16>:  0x0000c6f5  0x0000c6f5  0x0000c6f5  0x00000000
0x8240 <_vector_table+32>:  0x00000000  0x00000000  0x00000000  0x0000c4d5
0x8250 <_vector_table+48>:  0x0000c6f5  0x00000000  0x0000c48d  0x00024d3d
0x8260 <_irq_vector_table>: 0x0000c685  0x000195b5  0x0000c685  0x0000c685
0x8270 <_irq_vector_table+16>:  0x0000c685  0x0000c685  0x0000c685  0x0000c685
0x8280 <_irq_vector_table+32>:  0x0000c685  0x0000c685  0x0000c685  0x0000c685
0x8290 <_irq_vector_table+48>:  0x0000c685  0x0000c685  0x0000c685  0x0000c685
0x82a0 <_irq_vector_table+64>:  0x0000c685  0x0000c685  0x0000c685  0x0000c685
0x82b0 <_irq_vector_table+80>:  0x0000c685  0x0000c685  0x0000c685  0x0000c685
0x82c0 <_irq_vector_table+96>:  0x0000c685  0x0000c685  0x0000c685  0x0000c685
0x82d0 <_irq_vector_table+112>: 0x0000c685  0x0000c685  0x0000c685  0x0000c685
0x82e0 <_irq_vector_table+128>: 0x0000c685  0x0000c685  0x0000c685  0x0000c685
0x82f0 <_irq_vector_table+144>: 0x0000c685  0x0000c685  0x0000c685  0xffffffff
0x8300 <memchr>:    0x01fff001  0xdb2b2a10  0x0f07f010  0xf810d008
0x8310 <memchr+16>: 0x3a013b01  0xd02d428b  0x0f07f010  0xd1f6b342
0x8320 <memchr+32>: 0xea41b4f0  0xea412101  0xf0224101  0xf07f0407
0x8330 <memchr+48>: 0x23000700  0x5602e8f0  0xea853c08  0xea860501

This says that 0x8200 isn't the Vector Table, it should have been 0x8220. There is an offset error of 0x20 bytes!

VTOR Register should have been set to 0x8220 instead of 0x8200. By Arm specs, the VTOR Register must be aligned to 0x100 bytes... If the firmware tries to set it to 0x8220, it automagically becomes 0x8200, hence the error.

To continue debugging this, we need to set VTOR to 0x0, which contains a correct copy of our Vector Table. The gdb command is something like this (I need to check)...

*(0xE000ED08) = 0x0

We might have confused our Arm processor with our funny Vector Table. But I think I can patch this, hang on...

BTW thanks for hypnos-no-logging-debug-build and hypnos-no-logging-no-optimizations. Both of them were built for 0x0 (without bootloader) and they run fine on my PineTime. This shows that the firmware is OK, so I just need to patch the vector table correctly.

Stay tuned...

[lupyuen]

Yep Hypnos now works with MCUBoot! patch-bin.c now patches 2 copies of the Vector Table: at 0x0 and at 0x1e0, to compensate for the VTOR 0x20 byte offset.

https://github.com/lupyuen/patch-bin/blob/master/patch-bin.c

The files are here...

https://github.com/lupyuen/patch-bin/releases/tag/v1.0.0

The complete steps...

1. To patch zephyr.bin to zephyr-patched.bin:

   gcc -o patch-bin patch-bin.c
   ./patch-bin zephyr.bin zephyr-patched.bin zephyr-vector.bin

   The vector table is written to zephyr-vector.bin

2. To create MCUBoot firmware image zephyr-img.bin from the patched file zephyr-patched.bin:

   mcuboot/scripts/imgtool.py create --pad-header --align 4 --version 1.0.0 --header-size 32 --slot-size 475136 zephyr-patched.bin zephyr-img.bin
   mcuboot/scripts/imgtool.py verify zephyr-img.bin

   We include the "--pad-header" option because the file does not contain an empty MCUBoot header.

   See https://lupyuen.github.io/pinetime-rust-mynewt/articles/dfu#generate-a-firmware-image-file-for-pinetime

3. I skipped this step: To generate the DFU package zephyr-dfu.zip from the MCUBoot firmware image zephyr-img.bin:

   adafruit-nrfutil dfu genpkg --dev-type 0x0052 --application zephyr-img.bin zephyr-dfu.zip

   See https://lupyuen.github.io/pinetime-rust-mynewt/articles/cloud#create-dfu-package

4. I flashed the standard MCUBoot Bootloader to address 0x0...

   https://github.com/lupyuen/pinetime-rust-mynewt/releases/download/v4.1.7/mynewt_nosemi.elf.bin

   (Previously during debugging I flashed zephyr-vector.bin as the stub bootloader. But now I'm using the real MCUBoot Bootloader)

5. I flashed the imgtool output zephyr-img.bin to address 0x8000

Here's the demo with MCUBoot Bootloader and Hypnos...

https://youtu.be/-sCfmSxFHyw

[lupyuen]

Sorry for the Vector Table patching... Lemme explain how it got so complicated...

On Arm Cortex-M, it's common to have the firmware relocate the Vector Table (for various reasons). This means the firmware copies the Vector Table and sets the VTOR Register to point to the copy of the Vector Table.

Here's what I learned today...

Mynewt, FreeRTOS and Zephyr all handle Vector Table Relocation differently! And MCUBoot needs to handle all 3 types of Vector Table Relocation...

(1) Mynewt relocates the Vector Table to RAM 0x2000 0000

When Mynewt firmware boots, it always copies its Vector Table to 0x2000 0000. Then it sets the VTOR Register to 0x2000 0000.

Why relocate to RAM? I assume Mynewt allows the firmware to patch the Vector Table entries in RAM for custom interrupt handling.

So the MCUBoot Bootloader doesn't need to do anything for Mynewt... Mynewt Firmware fixes up the Vector Table itself, copying from 0x8020 to 0x2000 0000

(2) FreeRTOS doesn't relocate the Vector Table

Unfortunately MCUBoot has its own Vector Table. So MCUBoot needs to relocate the Vector Table on behalf of FreeRTOS.

According to our PineTime Bootloader Spec, the Vector Table is located at 0x8020. Now MCUBoot can't just point the VTOR Register to 0x8020, because VTOR must be aligned to 0x100.

Can we relocate to RAM like Mynewt? Nope because the RAM allocation is controlled by FreeRTOS. Our Vector Table may get clobbered by FreeRTOS later.

So I extended MCUBoot like this: Copy the Vector Table from 0x8020 to 0x7f00 in ROM. Then point VTOR to 0x7f00.

This is OK because 0x7f00 is aligned to 0x100, and all memory below 0x8000 is managed by the MCUBoot Bootloader.

So FreeRTOS starts, blissfully unaware that the Vector Table has been relocated to 0x7f00.

(3) Zephyr resets VTOR to its Vector Table in ROM

I thought the FreeRTOS workaround will work for Zephyr... Then I discovered today that Zephyr insists on setting VTOR to 0x8220 when it starts. Now this is not aligned to 0x100, so VTOR gets effectively set to 0x8200. Which contains garbage.

(Zephyr might be doing this to secure the firmware. Also relocating the Vector Table to RAM sounds vulnerable to attack)

So my workaround for Zephyr is to patch the firmware file to slide the Vector Table downwards by 0x20 bytes, to compensate for VTOR set to 0x8200. And it works! Everything else in the file stays the same, they don't slide.

Our patch-bin utility also copies the Vector Table to 0x8000. This needed because MCUBoot always looks at address 0x8024 for the start address of the firmware code, and jumps there.

[sdorre]

Hang on, Alignment i ssue sound exactly like what Endian posted here:

I just found this upstream commit related vector table alignment. It's just one month old, more recent than v2.3 that we're currently on.

So maybe we should try to do things right instead of patching blindly...

[lupyuen]

Pardon me if I take a little time to think things clearly... Because I fear I may patch things blindly ;-)

Now that we have established the root cause of the problem, I think I may have a solution that will...

(1) Not require any patching. I don't think Zephyr likes misaligned Vector Tables as much as Mynewt and FreeRTOS, so we shall put the Zephyr Vector Table at 0x8100. This means the MCUBoot Header shall occupy 0x8000 to 0x80ff.

(2) Work with existing Mynewt and FreeRTOS firmware without any patching. Mynewt and FreeRTOS shall stick to the same Vector Table at 0x8020. MCUBoot Header shall occupy 0x8000 to 0x801f.

(3) But requires a sight mod to the MCUBoot Bootloader, to be rolled out after the new batch of PineTime ships

(4) Meanwhile to make Hypnos work with the new batch of PineTime, we still need to make a patch (sorry!)

(5) But eventually no more patching to Hypnos, once the new MCUBoot Bootloader is fully regression tested and rolled out

But I shall keep everyone in suspense for a while :-) (For health reasons)

For the record: I think it's really good that you have allowed me to patch things blindly to make Hypnos work with the PineTime MCUBoot Bootloader, and I appreciate your patience and understanding (especially over the weekend). And now we're all better off understanding what's really happening (thanks to the patching). Yay!

This could be the first time that a wearable gadget supports multiple operating systems with a common bootloader. It's a great learning experience and I thank you all for that :-)

[endian-albin]

@lupyuen: The God of sleep salutes you! Now, enjoy a period of rest.

[JF002]

Thanks, @lupyuen for this detailed analysis!

(2) FreeRTOS doesn't relocate the Vector Table

Unfortunately MCUBoot has its own Vector Table. So MCUBoot needs to relocate the Vector Table on behalf of FreeRTOS.

In the case of InfiniTime, I don't think this behavior comes from FreeRTOS. Instead, I think it comes from linker script and startup code from NRF SDK.

I have full control on these files and can modify them if we decide that the management of the vector table should be modified for better interoperability.

[lupyuen]

Yep cool, I get nervous when we change too many things at once, so today let's focus on changing... (1) MCUBoot Bootloader (2) Hypnos Firmware While we keep InfiniTime (FreeRTOS) and Mynewt the same for now.

@endian-albin Would you be able to create a Hypnos Firmware zephyr.bin that starts at address 0x8100?

That means the Vector Table will be located at address 0x8100.

If this is feasible, then we add the MCUBoot Header at address 0x8000. This will be an MCUBoot Header with 0x100 bytes, instead of the usual 0x20 bytes.

This means that we shall call imgtool with header-size 256 instead of the usual 32 bytes...

mcuboot/scripts/imgtool.py create --pad-header --align 4 --version 1.0.0 --header-size 256 --slot-size 475136 zephyr.bin zephyr-img.bin
mcuboot/scripts/imgtool.py verify zephyr-img.bin

If you have problems exporting zephyr.bin in the right format: We can try removing the --pad-header option.

If the --pad-header option is removed, imgtool will overwrite the first 0x100 bytes of the file (instead of prepending 0x100 bytes)

Once you have the file, post it here and I'll test the modified MCUBoot that will support this new header size.

The MCUBoot Header Size is actually a field in the MCUBoot Header, so our MCUBoot Bootloader could read this field to figure out where the Vector Table is, without doing any patching. Thanks!

[endian-albin]

@lupyuen: I just pushed d7fdfd3 which has triggered this build. I will now prepare the zephyr-img.bin.

Update: You can now get zephyr.bin and zephyr-img.bin here.

[endian-albin]

@lupyuen: In the #63 branch I've now added the imgtool steps to the Github Actions workflow (in an unfinished, dirty way). You'll find the image under _temp/hypnos-mcuboot-app-img.bin in pinetime-hypnos-fw-4ab5f30.zip for example.

Update: I'm now using imgtool v1.6.0-rc2 referenced by the current Zephyr version. See pinetime-hypnos-fw-e2d9602.zip. Let me know if you need anything else.

Update 2: I used the wrong header size in the last build. This build should be good I hope.

[lupyuen]

Hmmm pinetime-hypnos-fw-8219954/build/zephyr/zephyr.map shows that the Vector Table is now at 0x8210 instead of our expected 0x8100...

                0x0000000000008010                _image_rom_start = 0x8010

rom_start       0x0000000000008010      0x2dc
                0x0000000000000200                . = 0x200
 *fill*         0x0000000000008010      0x200 
                0x0000000000008210                . = ALIGN (0x4)
                0x0000000000008210                _vector_start = .
 *(SORT_BY_ALIGNMENT(.exc_vector_table))
 *(SORT_BY_ALIGNMENT(.exc_vector_table.*))
 .exc_vector_table._vector_table_section
                0x0000000000008210       0x40 zephyr/arch/arch/arm/core/aarch32/cortex_m/libarch__arm__core__aarch32__cortex_m.a(vector_table.S.obj)
                0x0000000000008210                _vector_table
 *(SORT_BY_ALIGNMENT(.gnu.linkonce.irq_vector_table))
 .gnu.linkonce.irq_vector_table
                0x0000000000008250       0x9c zephyr/CMakeFiles/zephyr_final.dir/isr_tables.c.obj
                0x0000000000008250                _irq_vector_table
 *(SORT_BY_ALIGNMENT(.vectors))
                0x00000000000082ec                _vector_end = .

The memory configuration in the linker script looks unusual...

Memory Configuration

Name             Origin             Length             Attributes
FLASH            0x0000000000008010 0x0000000000071480 xr

Is this the default Zephyr linker script?

Could we change the start address to 0x8000 instead of 0x8010?

Could you also explain to me the changes you made to the original Zephyr config (i.e. Vector Table at 0x0) to create this firmware?

I'm wondering whether we might have added something extra that's causing this problem. Thanks!

BTW There's also a possibility that Zephyr really creates misaligned Vector Tables (like you mentioned). Though I find this odd, because Zephyr is supposed to work with MCUBoot for a while now...

https://github.com/zephyrproject-rtos/zephyr/commit/e80e655b01317458c5a85c38413f2cc4dbce1dc6

[endian-albin]

Hmmm pinetime-hypnos-fw-8219954/build/zephyr/zephyr.map shows that the Vector Table is now at 0x8210 instead of our expected 0x8100...

                0x0000000000008010                _image_rom_start = 0x8010

rom_start       0x0000000000008010      0x2dc

                0x0000000000000200                . = 0x200

 *fill*         0x0000000000008010      0x200 

                0x0000000000008210                . = ALIGN (0x4)

                0x0000000000008210                _vector_start = .

 *(SORT_BY_ALIGNMENT(.exc_vector_table))

 *(SORT_BY_ALIGNMENT(.exc_vector_table.*))

 .exc_vector_table._vector_table_section

                0x0000000000008210       0x40 zephyr/arch/arch/arm/core/aarch32/cortex_m/libarch__arm__core__aarch32__cortex_m.a(vector_table.S.obj)

                0x0000000000008210                _vector_table

 *(SORT_BY_ALIGNMENT(.gnu.linkonce.irq_vector_table))

 .gnu.linkonce.irq_vector_table

                0x0000000000008250       0x9c zephyr/CMakeFiles/zephyr_final.dir/isr_tables.c.obj

                0x0000000000008250                _irq_vector_table

 *(SORT_BY_ALIGNMENT(.vectors))

                0x00000000000082ec                _vector_end = .

The memory configuration in the linker script looks unusual...

Memory Configuration

Name             Origin             Length             Attributes

FLASH            0x0000000000008010 0x0000000000071480 xr

Is this the default Zephyr linker script?

Yes, based on the partition layout I suppose, which is defined in pinetime.dts. I've not used a custom script except once.

Could we change the start address to 0x8000 instead of 0x8010?

Yes, sure. I'll then move all the following partitions up by 10.

Could you also explain to me the changes you made to the original Zephyr config (i.e. Vector Table at 0x0) to create this firmware?

I once sent you two images built from master and not the dfu branch, which was a mistake, but you can see in the CI build history what branch a build is from, although I've forcepushed a lot so the entire history is not preserved.

I'm wondering whether we might have added something extra that's causing this problem. Thanks!

I'll try to make some variations in the configuration and generate new builds from each of them so that we can compare.

BTW There's also a possibility that Zephyr really creates misaligned Vector Tables (like you mentioned). Though I find this odd, because Zephyr is supposed to work with MCUBoot for a while now...

https://github.com/zephyrproject-rtos/zephyr/commit/e80e655b01317458c5a85c38413f2cc4dbce1dc6

I'll get back to you in about an hour.

[endian-albin]

@lupyuen: I've now built two more images, both with firmware offset at 0x8000 and one of them with the above-mentioned vector-table alignment commit cherry-picked to the v.2.3.0 version we're normally using. As @sdorre just mentioned in the PR, the build system automatically enables this option, however:

config TEXT_SECTION_OFFSET
    hex
    prompt "TEXT section offset" if !BOOTLOADER_MCUBOOT
    default 0x200 if BOOTLOADER_MCUBOOT
    default 0
    help
      If the application is built for chain-loading by a bootloader this
      variable is required to be set to value that leaves sufficient
      space between the beginning of the image and the start of the .text
      section to store an image header or any other metadata.
      In the particular case of the MCUboot bootloader this reserves enough
      space to store the image header, which should also meet vector table
      alignment requirements on most ARM targets, although some targets
      may require smaller or larger values.

I need the BOOTLOADER_MCUBOOT option for two things: to use the partition layout defined in pinetime.dts and the application function for marking the new image OK.

[endian-albin]

If I decrease all offsets by 0x200, then zephyr.map looks like this:

                0x0000000000007e10                _image_rom_start = 0x7e10

rom_start       0x0000000000007e10      0x3cc
                0x0000000000000200                . = 0x200
 *fill*         0x0000000000007e10      0x200 
                0x0000000000008010                . = ALIGN (0x4)
                0x0000000000008080                . = ALIGN (0x80)
 *fill*         0x0000000000008010       0x70 
                0x0000000000008100                . = ALIGN (0x100)
 *fill*         0x0000000000008080       0x80 
                0x0000000000008100                _vector_start = .
 *(SORT_BY_ALIGNMENT(.exc_vector_table))
 *(SORT_BY_ALIGNMENT(.exc_vector_table.*))
 .exc_vector_table._vector_table_section
                0x0000000000008100       0x40 zephyr/arch/arch/arm/core/aarch32/cortex_m/libarch__arm__core__aarch32__cortex_m.a(vector_table.S.obj)
                0x0000000000008100                _vector_table
 *(SORT_BY_ALIGNMENT(.gnu.linkonce.irq_vector_table))
 .gnu.linkonce.irq_vector_table
                0x0000000000008140       0x9c zephyr/CMakeFiles/zephyr_final.dir/isr_tables.c.obj
                0x0000000000008140                _irq_vector_table
 *(SORT_BY_ALIGNMENT(.vectors))
                0x00000000000081dc                _vector_end = .

You can get the CI build here.

As expected, decreasing the offset further by 0x100 generates:

               0x0000000000007e00                _image_rom_start = 0x7e00

rom_start       0x0000000000007e00      0x2dc
                0x0000000000000200                . = 0x200
 *fill*         0x0000000000007e00      0x200 
                0x0000000000008000                . = ALIGN (0x4)
                0x0000000000008000                . = ALIGN (0x80)
                0x0000000000008000                . = ALIGN (0x100)
                0x0000000000008000                _vector_start = .
 *(SORT_BY_ALIGNMENT(.exc_vector_table))
 *(SORT_BY_ALIGNMENT(.exc_vector_table.*))
 .exc_vector_table._vector_table_section
                0x0000000000008000       0x40 zephyr/arch/arch/arm/core/aarch32/cortex_m/libarch__arm__core__aarch32__cortex_m.a(vector_table.S.obj)
                0x0000000000008000                _vector_table
 *(SORT_BY_ALIGNMENT(.gnu.linkonce.irq_vector_table))
 .gnu.linkonce.irq_vector_table
                0x0000000000008040       0x9c zephyr/CMakeFiles/zephyr_final.dir/isr_tables.c.obj
                0x0000000000008040                _irq_vector_table
 *(SORT_BY_ALIGNMENT(.vectors))
                0x00000000000080dc                _vector_end = .

And here is the corresponding CI build.

[endian-albin]

No matter how I manipulate the offsets I'm unable to fulfil this requirement: 0x8020 onwards will contain the Interrupt Vector Table followed by the firmware code

If this is a hard requirement by the bootloader sent to the factory, then I suppose we can't avoid using a patched binary for this first version of the bootloader.

As you could see above, however, I can move the vector table to e.g. 0x8000, 0x8100 or 0x8200 for an updated version of the PineTime bootloader. @lupyuen, what _image_rom_start and _vector_table values do you suggest that we use for v2 of the bootloader?

[endian-albin]

Now we have something better I think!

@sdorre showed that we can force-set the text offset value by adding the following to Kconfig.pinetime:

config TEXT_SECTION_OFFSET
    hex
    prompt "TEXT section offset" if !BOOTLOADER_MCUBOOT
    default 0x20 if BOOTLOADER_MCUBOOT
    default 0
    help
      If the application is built for chain-loading by a bootloader this
      variable is required to be set to value that leaves sufficient
      space between the beginning of the image and the start of the .text
      section to store an image header or any other metadata.
      In the particular case of the MCUboot bootloader this reserves enough
      space to store the image header, which should also meet vector table
      alignment requirements on most ARM targets, although some targets
      may require smaller or larger values.

I did that and also set the partition layout for the internal flash memory to:

        boot_partition: partition@0 {
            label = "mcuboot";
            reg = <0x00000000 0x5DC0>;
        };
        slot0_partition: partition@8000 {
            label = "image-0";
            reg = <0x00008000 0x71480>;
        };
        scratch_partition: partition@79480  {
            label = "image-scratch";
            reg = <0x00079480 0x7640>;
        };

With v2.3.0, zephyr.map looks like this:

                0x0000000000008000                _image_rom_start = 0x8000

rom_start       0x0000000000008000       0xfc
                0x0000000000000020                . = 0x20
 *fill*         0x0000000000008000       0x20 
                0x0000000000008020                . = ALIGN (0x4)
                0x0000000000008020                _vector_start = .
 *(SORT_BY_ALIGNMENT(.exc_vector_table))
 *(SORT_BY_ALIGNMENT(.exc_vector_table.*))
 .exc_vector_table._vector_table_section
                0x0000000000008020       0x40 zephyr/arch/arch/arm/core/aarch32/cortex_m/libarch__arm__core__aarch32__cortex_m.a(vector_table.S.obj)
                0x0000000000008020                _vector_table
 *(SORT_BY_ALIGNMENT(.gnu.linkonce.irq_vector_table))
 .gnu.linkonce.irq_vector_table
                0x0000000000008060       0x9c zephyr/CMakeFiles/zephyr_final.dir/isr_tables.c.obj
                0x0000000000008060                _irq_vector_table
 *(SORT_BY_ALIGNMENT(.vectors))
                0x00000000000080fc                _vector_end = .
 *(SORT_BY_ALIGNMENT(.openocd_dbg))
 *(SORT_BY_ALIGNMENT(.openocd_dbg.*))

text            0x0000000000008100    0x26546
                0x0000000000008100                _image_text_start = .

Get the v2.3.0 build here.

With v2.3.0 + vector-table cherry-pick (merged into what will become v.2.4.0) zephyr.map instead becomes:

                0x0000000000008000                _image_rom_start = 0x8000

rom_start       0x0000000000008000      0x1dc
                0x0000000000000020                . = 0x20
 *fill*         0x0000000000008000       0x20 
                0x0000000000008020                . = ALIGN (0x4)
                0x0000000000008080                . = ALIGN (0x80)
 *fill*         0x0000000000008020       0x60 
                0x0000000000008100                . = ALIGN (0x100)
 *fill*         0x0000000000008080       0x80 
                0x0000000000008100                _vector_start = .
 *(SORT_BY_ALIGNMENT(.exc_vector_table))
 *(SORT_BY_ALIGNMENT(.exc_vector_table.*))
 .exc_vector_table._vector_table_section
                0x0000000000008100       0x40 zephyr/arch/arch/arm/core/aarch32/cortex_m/libarch__arm__core__aarch32__cortex_m.a(vector_table.S.obj)
                0x0000000000008100                _vector_table
 *(SORT_BY_ALIGNMENT(.gnu.linkonce.irq_vector_table))
 .gnu.linkonce.irq_vector_table
                0x0000000000008140       0x9c zephyr/CMakeFiles/zephyr_final.dir/isr_tables.c.obj
                0x0000000000008140                _irq_vector_table
 *(SORT_BY_ALIGNMENT(.vectors))
                0x00000000000081dc                _vector_end = .
 *(SORT_BY_ALIGNMENT(.openocd_dbg))
 *(SORT_BY_ALIGNMENT(.openocd_dbg.*))

text            0x00000000000081e0    0x26546
                0x00000000000081e0                _image_text_start = .

Get the "cherry-pick" build here.

[endian-albin]

Unfortunately, I was unable to boot the Hypnos image using the v2.3.0 build above. I followed the steps below but got stuck in a boot loop:

1. Installed the bootloader graphics from the InfiniTime repository
2. Installed mynewt_nosemi.elf.bin to 0x00
3. Installed the "img.bin" from the CI build to 0x8000 (caused a boot loop)
4. Used zephyr.bin and generated the "...img.bin" locally like this:

mcuboot/scripts/imgtool.py create --align 4 --version 1.0.0 --header-size 32 --slot-size 475136 --pad-header pinetime-mcuboot-app.bin pinetime-mcuboot-app-img.bin
mcuboot/scripts/imgtool.py verify pinetime-mcuboot-app-img.bin

(caused a boot loop)

[endian-albin]

@lupyuen: I'm sorry to bombard you with so many posts but here I try to answer a question directly.

@endian-albin Would you be able to create a Hypnos Firmware zephyr.bin that starts at address 0x8100?

That means the Vector Table will be located at address 0x8100.

If this is feasible, then we add the MCUBoot Header at address 0x8000. This will be an MCUBoot Header with 0x100 bytes, instead of the usual 0x20 bytes.

This means that we shall call imgtool with header-size 256 instead of the usual 32 bytes...

mcuboot/scripts/imgtool.py create --pad-header --align 4 --version 1.0.0 --header-size 256 --slot-size 475136 zephyr.bin zephyr-img.bin mcuboot/scripts/imgtool.py verify zephyr-img.bin

If you have problems exporting zephyr.bin in the right format: We can try removing the --pad-header option.

If the --pad-header option is removed, imgtool will overwrite the first 0x100 bytes of the file (instead of prepending 0x100 bytes)

Once you have the file, post it here and I'll test the modified MCUBoot that will support this new header size.

The MCUBoot Header Size is actually a field in the MCUBoot Header, so our MCUBoot Bootloader could read this field to figure out where the Vector Table is, without doing any patching. Thanks!

I've tried to do as you suggested and will go through each of the steps below. You can get zephyr-cherry.zip (containing many build files) and pinetime-hypnos-cherry-mcuboot-app-img.bin here.

1. Modify Kconfig.pinetime so it looks like this:

   
   menu "Pinetime"

config TEXT_SECTION_OFFSET hex prompt "TEXT section offset" if !BOOTLOADER_MCUBOOT default 0x100 if BOOTLOADER_MCUBOOT # changed from 0x200 default 0 help If the application is built for chain-loading by a bootloader this variable is required to be set to value that leaves sufficient space between the beginning of the image and the start of the .text section to store an image header or any other metadata. In the particular case of the MCUboot bootloader this reserves enough space to store the image header, which should also meet vector table alignment requirements on most ARM targets, although some targets may require smaller or larger values.

rsource "drivers/sensor/Kconfig" rsource "subsys/Kconfig"

endmenu

2. Declare the partition layout for internal flash like this in `pinetime.dts` (offsets and partition sizes are both represented in hex):

    boot_partition: partition@0 {
        label = "mcuboot";
        reg = <0x00000000 0x5DC0>;
    };
    slot0_partition: partition@8000 {
        label = "image-0";
        reg = <0x00008000 0x71480>;
    };
    scratch_partition: partition@79480  {
        label = "image-scratch";
        reg = <0x00079480 0x7640>;
    };

3.  cherry-pick commit 8d29660 from master
> arch: arm: cortex_m: align vector table based on VTOR requirements
>     
> Enforce VTOR table offset alignment requirements on Cortex-M
> vector table start address

4. Run `west build -p -b pinetime hypnos`

5. Run the following imgtool commands:

./imgtool.py create --align 4 --version 1.0.0 --header-size 256 --slot-size 475136 zephyr.bin pinetime-hypnos-cherry-mcuboot-app-img.bin ./imgtool.py verify pinetime-hypnos-cherry-mcuboot-app-img.bin

I did remove the `--pad-header` option so as to overwrite the beginning of the file, because here `zephyr.map` looks like this:

            0x0000000000008000                _image_rom_start = 0x8000

rom_start 0x0000000000008000 0x1dc 0x0000000000000100 . = 0x100 fill 0x0000000000008000 0x100 0x0000000000008100 . = ALIGN (0x4) 0x0000000000008100 . = ALIGN (0x80) 0x0000000000008100 . = ALIGN (0x100) 0x0000000000008100 _vector_start = . (SORT_BY_ALIGNMENT(.exc_vector_table)) (SORT_BY_ALIGNMENT(.exc_vector_table.)) .exc_vector_table._vector_table_section 0x0000000000008100 0x40 zephyr/arch/arch/arm/core/aarch32/cortex_m/libarcharmcoreaarch32cortex_m.a(vector_table.S.obj) 0x0000000000008100 _vector_table (SORT_BY_ALIGNMENT(.gnu.linkonce.irq_vector_table)) .gnu.linkonce.irq_vector_table 0x0000000000008140 0x9c zephyr/CMakeFiles/zephyr_final.dir/isr_tables.c.obj 0x0000000000008140 _irq_vector_table (SORT_BY_ALIGNMENT(.vectors)) 0x00000000000081dc _vector_end = . (SORT_BY_ALIGNMENT(.openocd_dbg)) (SORT_BY_ALIGNMENT(.openocd_dbg.))

text 0x00000000000081e0 0x26546 0x00000000000081e0 _image_text_start = .

I don't have access to your new, work-in-progress bootloader so I couldn't try it out myself. Let me know if there is anything more I can do.