RFC: Precalculated swap log for supporting devices with bigger write block sizes

de-nordic commented 1 week ago

Where we are

At this point MCUboot, while moving sectors around, logs status of operations with three flags per sector. With devices with write block size of 4b it amounts to 12 bytes per moved sector. This becomes problematic when you want to make MCUboot work with devices that have bigger write block sizes; for example with devices that require 8 bytes this is already 24 bytes, and becomes 48 for devices with 16 byte write block size. When you get to devices that have blocks like 512 byte, this become hard to handle.

Note: while using "sector" here I mean logical sector of N * erase-block-size of device with the biggest erase-block-size.

What can we do?

The proposal here is to replace the swap log with information block, layout record, that would state desired state of slot and swap algorithm would just try to restore the desired state from what it has on flash.

The layout record could consist of array of sha of sectors in order they should appear in slot. Note that sha256 is 32 byte long so it is already shorter then log entry for 16 byte write-block-size device.

For example

    [ State header ]
    [ sha(sector0) ]
    [ sha(sector1) ]
    ...
    ...
    ...
   [ sha(sectorN) ]
   [ sha(layout record) ]

Where sha is whatever sha we select (could be configurable, sha256 seems fine), and sectorX i logical sector size block on device, within slot. Each slot will have its own expected layout record, so there would be two such blocks. The layout record sha is just to make sure that we the record is complete.

Note that, if we put aside header size, the log size is now just N * sizeof(sha) and is aligned to write-block-size as a whole not per log entry (actually it will be aligned to erase block size, or logical sector).

Unresolved questions:

1) Where do we store this record? Probably storing logs at the end of slot is not a good idea and we should have reserved space outside of slots and the entire slot should be dedicated to application image; problem will be here with devices that have big erase sizes as the advantage of small log record would be killed by using entire large independent erase block size just to store it, and it is better for such devices to have that record inside slot. This could be probably configurable depending on device layout. 2) How to mark test/confirm. Generally if we just assume that MCUboot always attempts to restore layout according to the pre-defined log, then just appearance of the log would mean that the swap should happen, and confirmation could be just a removal of the log - this is again problem for devices with big erase blocks, so they would have to mark somehow that they wan to stay with tested layout anyway. 3) Should application carry its own layout record? Probably, specifically when signed. Also calculating record on device is risky, but should be evaluated.

@nvlsianpu @d3zd3z @nordicjm @butok @erwango

butok commented 5 days ago

To clarify. Will these sha be generated and preinstalled by imagetool, or during run-time?

de-nordic commented 5 days ago

To clarify. Will these sha be generated and preinstalled by imagetool, or during run-time?

Actually that is point 3 in unresolved question. I guess that we can have it following ways, probably configurable: 1) the sha table is calculated and carried by every image + we have it pre calculated, we are sure that it is not manipulated by hardware errors +/- kinda saves time on calculating but we mostly have accelerators to do that job anyway - takes space with image 2) calculated on hardware: + saves image space +/- time of calculation may depend on having accelerator - risk of calculating something wrong 3) combo: sha of each page not pre-calculated but sha of the sha table is + saves image space +/- time of calculation may depend on having accelerator + even if we have calculated something wrong, or the are where we calculate the sha is broken, the image carried sha of sha table will prevent us from bricking device

nordicjm commented 5 days ago

I don't see how it could be done without having the hash table on the flash, if a device is rebooting during a page being swapped, how are you going to know which part goes with which partial image? One sector will be corrupted. If you don't have a checksum/hash of the sectors for each image, you can't know where that is, nor can you re-generate an "in RAM" hash list because then you have a huge number of operations to try and put a bunch of hashes together to come up with the "sha of the sha table"

de-nordic commented 4 days ago

I don't see how it could be done without having the hash table on the flash, if a device is rebooting during a page being swapped, how are you going to know which part goes with which partial image? One sector will be corrupted. If you don't have a checksum/hash of the sectors for each image, you can't know where that is, nor can you re-generate an "in RAM" hash list because then you have a huge number of operations to try and put a bunch of hashes together to come up with the "sha of the sha table"

No, we are intended to have it on flash, the question is where we get it from when we start the swap (and then where we place it).

mcu-tools / mcuboot