flash space coordination so it allows LCM to grow

HomeACcessoryKid commented 5 years ago

This issue serves to collect user feedback, so this is your chance to influence what will happen...

The issue: current code (0.9.13) leaves only 1000 bytes between the main routine ending in sector 0xf4000 and the lower_cert_sector at 0xf5000.

I cannot introduce any new code of any value without running into the lower_cert_sector.

HomeACcessoryKid commented 5 years ago

PROPOSAL1 (not coded anything of this yet...)

I will move the lower_cert_sector to 0x8b000 instead of the current 0xf5000 allowing some small updates.
If LCM needs to grow even more, I should reclaim 0x8c000.
Or a more complicated exercise of shrinking sys_params which I hope to avoid.
Homekit code (the typical deployments currently out there) should actively try to use the sysparam system so the higher_cert_sector can move from 0xf6000 to 0x8c000 in the future if more space is needed to grow.
The official approach is that user applications are discouraged from using anything between user code and LCM space

In order to coordinate between user code and LCM, two more sysparams are introduced:

ota_user_space_min which will contain the first address at which user related info is stored.
ota_user_space_max which will contain the first address at which nothing user related is stored.

These are intended to be at a sector border.
If these parameters are not already defined, they will be set as follows:

ota_user_space_min: the sector where LCM_space begins
ota_user_space_max: the sector after where the user_main file downloaded.

Before LCM will update to a higher version, it will collect a value from the GitHub server, predicting the sizes needed after updating LCM and if this means either ota_main or ota_boot overlaps with ota_user_space it will not update itself anymore, only user_main (if that still fits!) Users can set these parameters to contain a different values if they want to manipulate the flash use by blocking an ota update until they have made sure they are ready for the updated sizes.

HomeACcessoryKid commented 5 years ago

PROPOSAL2 (not coded anything of this either)

Much the same as proposal1 BUT I would immediately transplant any existing sys_param area to 0x8d000 (2 sectors) and put the certificate sectors at 0xf9 and 0fa.

This creates two sectors to grow if needed
if the sysparam area needs to be grown then there would be another parameter ota_sysparam_size that would indicate how much space to use for it. If this is set, it implies that the HomeKit sector 0x8c000 is free to use and it should be up to LCM to make the expansion and transfer the existing data to the new sysparam situation

User code HAS TO USE sys_param_init starting from a low value to see where the parameters have ended up this time.

Mayor issue is that existing user code does not use sys_param_init yet and we need to signal if the user code is ready for this. If not, there is a minimum version of OTA that supports the old situation and without a flag to show that the user code is ready for it, LCM will not update itself beyond this version.

HomeACcessoryKid commented 5 years ago

PROPOSAL3 (not coded anything of this)

Convert the 4 sectors of sys_param at 0xf7000 into 2 sectors at the same 0xf7000. The certificate sectors can then go into 0xf9000 and 0xfa000 AND the user app code doesn't need any change AND it creates 2 sectors for growth right now.

The three new sys_params will be introduced as in prop1 and prop2 If the LCM does not work anymore within this available space, the user code will set a flag ota_lcm2 and this will switch over to a new repository that works based on proposal2 which includes proposal1

I think that with this we can have short term solution with the essential space for some needed updates and a long term strategy just in case.

HomeACcessoryKid commented 5 years ago

Latest update is that proposal3 seems the way to go. I have however one dilemma:

In the (unlikely?) case that a device is already using a lot of sysparam space and that after compacting it still doesn't fit in 3KB (or even 4KB) then I see no way to save this device from data-loss.

Considering that using this software was a 'no guarantee' job and the estimation that close to no devices will be affected, I will proceed anyhow, without fallback.

This is your place to comment if you do not agree

HomeACcessoryKid commented 5 years ago

option 3 has been implemented and it seems to work fine If you do have an issue with reduced sysparam size, do make a remark and I'll start implementing the remaining features of proposal 2 and 1

HomeACcessoryKid / life-cycle-manager

flash space coordination so it allows LCM to grow #4