de-nordic
commented
1 year ago Current partition definitions provide us most basic and versatile configuration for starting simple application or building simpler samples from Zephyr tree, for defined used DK boards. It basically allows you to start developing by extending "Hello world" + enabling MCUboot and maybe using LittleFS on storage focusing on software, before you need to go down the DTS definitions to tune partitions to your actual needs. It is in most cases too basic or inadequate for application development and users will have to modify DK definition to address/simulate their target board.
Inconsistency in the partition definitions.
As noted above, the definition of Flash device partitions is somewhat "inconsistent" (overlay for tests or in board device tree).
Tests will have overlays because they target specific areas, they need specific configuration to do so, and very often are only to be run on some representative set of boards. You may have Flash API tests that by default target internal flash and are able to work on most of boards, and will use storage_partition as target for API tests. These tests may be easily moved to test external flash, when partition is moved to external storage, and that is done with overlay. Not a best approach probably but allows to use the same code addresing storage_partition to test SoC and SPI device just by using different overlay.
Some SoC's have on-silicon flash memory (like NXP's lpc family). Having the partitions defined in the board device tree is as inappropriate as in the SoC dtsi....
OK, I do not understand why this is inappropriate, as you have to divide the flash for various purposes when you develop application and build system needs to know where your app lands. When you use MCUboot, it needs to know what to boot. Those things need to be decided before you start to deploy your software, and basically describe hardware of your board and how memory is reserved. Very often you will not be able to change these settings one you start deploying boards, because you can not just start building your app with new DTS layout of partitions, because for example your bootloader may get confused, or your new app may start damaging some data stored based on previous layout. And again, the definitions provided within tree are just for convenience of making it easier to write samples or start developing applications.
The same will apply for eMMC devices. In other words, the Zephyr workflow should enable the customer's/end user's tools for manufacturing. the end user's/silicon vendors could write west extensions to generate the file(s) needed for their respective tools.
There is Kconfig option that allows you to enable custom Flash Map definitions, so you are free to do that. It is not easy though because changes here affect build system and os internals, where Zephyr tries to, at compile time, figure out sizes of partitions, offset and devices they are placed on.
This assumes the "system" data is easily accessible, perhaps in a 'system.yaml' file.
You can just generate DT partition overlay from your yaml, vendors can have any tooling they want, as long as they are generating something that can be used with Zephyr.
Introduction
Flash memory devices are usually "partitiioned" into logical segments, each with a specific purpose, e.g. storage, boot loader, application executables.
Partitions are sometimes specified in the board device tree (partitions are not board specific but "system" specific or in a device tree overlay to meet the requirements of a test (e.g. tests/subsys/fs/littlefs).
It should be recognized partitions are either application specific or system specific (e.g. MCUboot + application)
Overall goal of this RFC is to start exploring how we can adjust the Zephyr workflow to:
Problem description
Inconsistency in the partition definitions.
As noted above, the definition of Flash device partitions is somewhat "inconsistent" (overlay for tests or in board device tree).
Some SoC's have on-silicon flash memory (like NXP's lpc family). Having the partitions defined in the board device tree is as inappropriate as in the SoC dtsi....
Manufacturing wise, images have to be prepared prior to "flashing". The partition information is needed to write the binary blobs (e.g. executables, data, etc) to the proper offsets. The same will apply for eMMC devices. In other words, the Zephyr workflow should enable the customer's/end user's tools for manufacturing. the end user's/silicon vendors could write west extensions to generate the file(s) needed for
their respective tools. This assumes the "system" data is easily accessible, perhaps in a 'system.yaml' file.
Proposed change
Move the partition definitions into a device tree overlay, start managing 'system' bits (partitions and later perhaps signing) in a 'system.yaml' file.
Detailed RFC
In this section of the document the target audience is tjhe dev team. Upon reading this section each engineer should have a rather clear picture of what needs to be done in order to implement the described feature.
Proposed change (Detailed)
System definitions
System definitions cover:
Other system wide definitions (not yet fleshed out...):
Workflow changes
Generating images can be achieved in different ways:
The best approach to supporting these tools is to rely on west extensions, written by silicon vendors/end users to generate the files needed by their respective tools. It is most useful if the system information needed is located in a single file, i.e. system.yaml. It makes writing extensions "simpler". The details of the CLI for that extension is left to the Si-vendor/end user.
When building an application for a specific core+board, the partition overlay can be applied when the device tree is generated
Dependencies
west, extension to extract system information bits required for tools to generate images, etc.
Concerns and Unresolved Questions
Unsure about the impact on sysbuild
Alternatives
Do nothing and keep the current workflow, keeping the lack of flexibility.