tfm: stack limitation on unsecured application on b_u585i_iot02a_ns

[RomainPelletant]

Describe the bug When using TFM (with STM32U585), the stack is limited on non-secure application.

First test based on tfm_ipc sample:

• Add several threads in tfm_ipc sample (more than 132KB as total stack)
• Run tfm_ipc sample : reboot looping after TF-M boot (non secure boot)

Second test based on tfm_ipc sample under stack limit:

• Same as first test but limit total stack to 131KB
• Run successfully

Third test without TFM:

• Remove all TFM-related from TFM_IPC sample
• Add several threads (more than 200KB as total stack)
• Run succesfully

Please also mention any information which could help others to understand the problem you're facing:

• What target platform are you using? STM32U585 custom board
• What have you tried to diagnose or workaround this issue? I tried to increase NS_STACK_SIZE in region_defs.h but no success. Maybe MSPLIM/PSPLIM registers could trigger stack overflow/protection. The stack limit reached is equivalent to memory shift by an offset equal to 0x20000 (131072d)
• Is this a regression? If yes, have you been able to "git bisect" it to a specific commit? Tested with TF-M 1.7 and TF-M 1.8 with the same result Zephyr 3.4.99 (rebased on main) MCU: STM32U585QI

To Reproduce Steps to reproduce the behavior:

1. Open sample TFM_IPC (in sample/tfm_integration/tfm_ipc)
2. Increase main stack size and add K_THREAD_DEFINE to reach up to 131KB of stacks
3. Run with TFM
4. make & flash (regression.sh && west flash)
5. See error

Expected behavior MCU has 768KB of RAM and I expect to use them, at least 256KB.

Impact showstopper

Logs and console output

[Sec Thread] Secure image initializing!
Booting TF-M v1.8.0
[INF][Crypto] Initialising HW accelerator... complete.
[WRN] This device was provisioned with dummy keys. This device is NOT SECURE
[Sec Thread] Secure image initializing!
Booting TF-M v1.8.0
[INF][Crypto] Initialising HW accelerator... complete.
[WRN] This device was provisioned with dummy keys. This device is NOT SECURE
[Sec Thread] Secure image initializing!
Booting TF-M v1.8.0
[INF][Crypto] Initialising HW accelerator... complete.
[WRN] This device was provisioned with dummy keys. This device is NOT SECURE
[Sec Thread] Secure image initializing!
Booting TF-M v1.8.0
[INF][Crypto] Initialising HW accelerator... complete.

Environment (please complete the following information):

• OS: MacOS
• Zephyr SDK 0.15.2
• Zephyr main for test (TF-M 1.8)

[RomainPelletant]

To reproduce with TF-M 1.8, it seems like build is broken (as reported in #61503) :

Issue comes from MCUBoot (commit 5b1d511 & 3016d00) introducing a new function prototype (in boot_record.h)

int boot_save_shared_data(const struct image_header *hdr, const struct flash_area *fap,
                          const uint8_t slot, const int max_app_size)

with the same function name of another source file already present in tfm/bl2/src/shared_data.c

int boot_save_shared_data(const struct image_header *hdr,
                          const struct flash_area *fap)

As a workaround for test purpose, I disabled boot measurement capabilities.

[RomainPelletant]

Someone has an idea about that issue? Maybe a direction ? @erwango FYI

[erwango]

@RomainPelletant Would you be able to provide a map file for each of the 2 first cases ?

[erwango]

@RomainPelletant Here are some hints: RAM partition is done as follows on U575:

– SRAM1: three 64-Kbyte blocks (total 192 Kbytes)
– SRAM2: 8-Kbyte + 56-Kbyte blocks (total 64 Kbytes). In addition SRAM2 blocks
can be retained in Standby mode.
– SRAM3: eight 64-Kbyte blocks (total 512 Kbytes)
– SRAM4: 16-Kbyte

TFM uses SRAM2, while Zephyr NS application is set to use SRAM1 defined as a continuous RAM of 768K. So once NS starts using SRAM2, which is TFM assigned area, a reset is triggered (which is, I guess an expected TF-M behavior).

Would you be able to try to get Zephyr NS application to use SRAM3 ?

[RomainPelletant]

@erwango thank you so much for your help! I will give it a try and come back to you (with map files if the issue is not solved)

[erwango]

@RomainPelletant Did you had a chance to try this lead ?

[RomainPelletant]

@erwango For now, the device is unable to boot at all with the modification. I think the mistake comes from me. I am trying few fixes and come back to you

[RomainPelletant]

@erwango Board is available on next Monday. will try it again

[ROMAINPC]

Hi, I'm continuing to debug this issue.

To be exact, this issue is triggered with a Zephyr total Non-Secure RAM usage of 128Ko. I was able to extend the available RAM to 192Ko (the default SRAM1 size) by removing the use of #define S_TOTAL_RAM1_SIZE (0x10000) in region_defs.h file.\ Manipulating the other constants in this file prevents the system from booting. No console output.\ So I can't use SRAM3 instead of SRAM1 simply by modifying the RAM base address for the non-secure state.\ Indeed, the function gtzc_init_cfg() in our TF-M external zephyr module explicitly defines the partitioning of SRAMs. And still rely on some #defines in our region_defs.h.

According to the STM32U585xx datasheet it is possible to configure every addresses in all SRAMs, as secure or not or Non-Secure Callable. Two virtual addresses spaces are used to refer to them. (0x2000 0000 and 0x3000 0000 ).\ However the TF-M and its secure boot mechanism imposes a default architecture with SRAM1 Non-Secure and SRAM2 Secure plus 32Ko in SRAM3. (cf. page 30 of ST UM2851). NB : There no longer appear to be any NSC addresses.

It seems inappropriate and complex to me to modify the TF-M to use SRAM3. Continuously (total 192+512) or to replace SRAM1.\ So I still want to tell Zephyr to load in SRAM3.\ Since we must not modify NS_RAM_ALIAS_BASE in region_defs.h, how to do this from the Zephyr project ? Thanks in advance.

[ROMAINPC]

We found a solution, without touching the TF-M sources.\ With a modification in our device tree, it shifts the RAM base address used by Zephyr :

/ {
       // ...
       chosen {
            // ...
            zephyr,sram = &sram3;
            // ...
       }

    sram3: memory@20040000 {
        reg = <0x20040000 DT_SIZE_K(512)>;
        };
}

I also looked into using Zephyr code relocation to be able to use on demand SRAM1 again. In case SRAM3 still not enough. Can we relocate the heap and the stack with this feature ? By adding a linker.ld file to our board with a new MEMORY block, I'm able to relocate files inside SRAM1. However at execution only few bytes are stored inside.

[erwango]

We found a solution, without touching the TF-M sources.

Great news. It could be interesting to push this upstream, if that doesn't bring too much drawbacks/side effects (I see none at first sight, but this might require some deeper thinking).

I also looked into using Zephyr code relocation to be able to use on demand SRAM1 again. In case SRAM3 still not enough. Can we relocate the heap and the stack with this feature ?

You might be interested by https://github.com/zephyrproject-rtos/zephyr/pull/63014

[erwango]

@ROMAINPC what are the next steps ?

[github-actions[bot]]

This issue has been marked as stale because it has been open (more than) 60 days with no activity. Remove the stale label or add a comment saying that you would like to have the label removed otherwise this issue will automatically be closed in 14 days. Note, that you can always re-open a closed issue at any time.