HSS not booting in Factory Secure Boot Mode 3

nearly-big-endian commented 1 year ago

Hi,

We are currently using latest HSS (0.99.33, release 2022.10) together with SoftConsole 2022.2 on an IcicleKit. When programmed in Non Secure Boot Mode 1 (via SoftConsole), HSS boots successfully and proceeds with the load of u-boot then our Linux image.

When programming the same HSS binary in Factory Secure Boot Mode 3 (still through SoftConsole), HSS hangs at the Memory testing stage (as logs show below).

The root cause does not seem to be the DDR testing itself though, as disabling CONFIG_MEMTEST just leads the HSS to hang at a later stage. It's more looking like a memory mapping issue.

Any idea what is going on ?

Some additional notes:

this is using default HSS configuration (for mpfs-icicle-kit-es board)
the linker script was not changed (using original hss-l2scratch.lds)
it does not make a difference whether the HSS image is signed with auto-generated keys or externally provided keys
security is NOT activated on that board (no user keys or security settings were set in liberoSoC)

HSS Boot log in Non-Secure Boot Mode 1 (succeeds):

[6.238558] PolarFire(R) SoC Hart Software Services (HSS) - version 0.99.33-dev-build
MPFS HAL version 2.0.101 / DDR Driver version 0.4.018 / Mi-V IHC version 0.1.1 / BOARD=mpfs-icicle-kit-es
(c) Copyright 2017-2022 Microchip FPGA Embedded Systems Solutions.

incorporating OpenSBI - version 1.0
(c) Copyright 2019-2022 Western Digital Corporation.

[6.274077] Build ID: 60f067913789680ee6a23d010d5d0c0629633be4
[6.281620] Built with the following tools:
 - riscv64-unknown-elf-gcc (xPack GNU RISC-V Embedded GCC (Microsemi SoftConsole build), 64-bit) 8.3.0
 - GNU ld (xPack GNU RISC-V Embedded GCC (Microsemi SoftConsole build), 64-bit) 2.32

[6.305680] NOTICE: Running from L2 Scratchpad

[6.311887] Serial Number:
5995625459b1187379e72b44fa110c2100000000000000000000000000000000000000000000000000000000000000000000
[6.325827] Segment Configuration:
        Cached: SEG0_0: offset 0x0080000000, physical DDR 0x00000000
        Cached: SEG0_1: offset 0x1000000000, physical DDR 0x02000000
    Non-cached: SEG1_2: offset 0x00c0000000, physical DDR 0x78000000
Non-cached WCB: SEG1_4: offset 0x00d0000000, physical DDR 0x78000000
[6.357431] L2 Cache Configuration:
    L2-Scratchpad:  4 ways (512 KiB)
         L2-Cache:  8 ways (1024 KiB)
           L2-LIM:  4 ways (512 KiB)
[6.373376] DDR-Lo size is   32 MiB
[6.378054] DDR-Hi size is 1888 MiB
[6.382733] Please ensure that jumpers J34/J43 are correct for 1.8V MMC voltage...
[6.392281] Attempting to select SDCARD ... Failed
[6.450341] Attempting to select eMMC ... Passed
Press a key to enter CLI, ESC to skip
Timeout in 1 second
..
[7.803883] CLI boot interrupt timeout
[7.808848] Initializing Mi-V IHC
[7.813336] Initializing IPI Queues (6056 bytes @ 0xa02cf20)...
[7.820688] Initializing PMPs
[7.824793] Initializing Boot Image ...
[7.829854] Trying to boot via MMC ...
[7.834819] Preparing to copy from MMC to DDR ...
[7.841711] Validated GPT Header ...
[7.863107] Validated GPT Partition Entries ...
[7.869079] Boot Partition found at index 1
[7.874556] Attempting to read image header (1632 bytes) ...
[7.884251] Copying 691440 bytes to 0xa0000000
[7.898790] MMC: Boot Image registered ...
[7.904207] Boot image passed CRC
[7.908981] Boot image set name: "PolarFire-SoC-HSS::U-Boot"
<<< boot continues >>>

HSS Boot log in Factory-Secure Boot Mode 3 (fails):

[6.237898] PolarFire(R) SoC Hart Software Services (HSS) - version 0.99.33-dev-build
MPFS HAL version 2.0.101 / DDR Driver version 0.4.018 / Mi-V IHC version 0.1.1 / BOARD=mpfs-icicle-kit-es
(c) Copyright 2017-2022 Microchip FPGA Embedded Systems Solutions.

incorporating OpenSBI - version 1.0
(c) Copyright 2019-2022 Western Digital Corporation.

[6.273416] Build ID: 60f067913789680ee6a23d010d5d0c0629633be4
[6.280959] Built with the following tools:
 - riscv64-unknown-elf-gcc (xPack GNU RISC-V Embedded GCC (Microsemi SoftConsole build), 64-bit) 8.3.0
 - GNU ld (xPack GNU RISC-V Embedded GCC (Microsemi SoftConsole build), 64-bit) 2.32

[6.305020] NOTICE: Running from L2 Scratchpad

[6.311226] Serial Number:
5995625459b1187379e72b44fa110c2100000000000000000000000000000000000000000000000000000000000000000000
[6.325166] Segment Configuration:
        Cached: SEG0_0: offset 0x0080000000, physical DDR 0x00000000
        Cached: SEG0_1: offset 0x1000000000, physical DDR 0x02000000
    Non-cached: SEG1_2: offset 0x00c0000000, physical DDR 0x78000000
Non-cached WCB: SEG1_4: offset 0x00d0000000, physical DDR 0x78000000
[6.356770] L2 Cache Configuration:
    L2-Scratchpad:  4 ways (512 KiB)
         L2-Cache:  8 ways (1024 KiB)
           L2-LIM:  4 ways (512 KiB)
[6.372715] DDR-Lo size is    0 MiB
<<< HSS hangs here >>>

mpfsBootmodeProgrammer boot log (for boot mode 3):

18:01:04 DEBUG - Looking for ELF file in the work directory.
18:01:04 DEBUG - ELF file found: "hss-envm-wrapper.elf".
18:01:04 INFO  - Selected boot mode "3 - factory secure boot from eNVM" and working in directory "c:\Microchip\hart-software-services\Default".
18:01:04 DEBUG - Workdir=c:\Microchip\hart-software-services\Default die=MPFS250T_ES diePackage=FCVG484 bm=3 - factory secure boot from eNVM verify=true elf=hss-envm-wrapper.elf
18:01:04 DEBUG - Invoking command: "C:/Microchip/Libero_SoC_v2022.2/Designer/bin64//fpgenprog.exe new_project --location c:\Microchip\hart-software-services\Default/bootmode3/fpgenprogProject --target_die MPFS250T_ES --target_package FCVG484".
18:01:04 DEBUG - Finished with exit code: "0".
18:01:04 INFO  - Generating BIN file...
18:01:04 DEBUG - Invoking command: "C:\Microchip\SoftConsole-v2022.2\/riscv-unknown-elf-gcc/bin/riscv64-unknown-elf-objcopy.exe --version".
18:01:04 DEBUG - Finished with exit code: "0".
18:01:04 DEBUG - Invoking command: "C:\Microchip\SoftConsole-v2022.2\/riscv-unknown-elf-gcc/bin/riscv64-unknown-elf-objcopy.exe -O binary c:\Microchip\hart-software-services\Default/hss-envm-wrapper.elf c:\Microchip\hart-software-services\Default/bootmode3/hss-envm-wrapper.bin ".
18:01:04 DEBUG - Finished with exit code: "0".
18:01:04 DEBUG - Size of the BIN file is "95184 (decimal)" or "0x000173D0 (hex)".
18:01:04 DEBUG - Invoking command: "C:\Microchip\SoftConsole-v2022.2\/riscv-unknown-elf-gcc/bin/riscv64-unknown-elf-readelf.exe --program-headers c:\Microchip\hart-software-services\Default/hss-envm-wrapper.elf ".
18:01:05 DEBUG - Finished with exit code: "0".
18:01:05 DEBUG - Entry point 0x20220100
18:01:05 DEBUG - The detected entry address is "20220100".
18:01:05 INFO  - Generating SBIC (Secure Boot Image Certificate)...
18:01:05 INFO  - Generating ECDSA NIST P-384 keys...
18:01:05 DEBUG - Public key X(hex)=0x36a32b90aed78a26fef24130e019dfbb819bb43a1d93446977fbc41a585996ca7fbfbbb917a90600db941a687a4c26b5
18:01:05 DEBUG - Public key Y(hex)=0xebcdbbfea25d1c2973492ff97a02bf59446229218d6ea8287dbe656073e3ada5b85fcedf968430a551da3509bc1d21bb
18:01:05 INFO  - Generating HEX file...
18:01:05 DEBUG - Invoking command: "C:\Microchip\SoftConsole-v2022.2\/riscv-unknown-elf-gcc/bin/riscv64-unknown-elf-objcopy.exe -I binary -O ihex --change-section-lma *+0x20220000 c:\Microchip\hart-software-services\Default/bootmode3/hss-envm-wrapper-bm3-p0.bin c:\Microchip\hart-software-services\Default/bootmode3/hss-envm-wrapper-bm3-p0.hex ".
18:01:05 DEBUG - Finished with exit code: "0".
18:01:05 INFO  - Preparing for bitstream generation...
18:01:05 DEBUG - Getting the SBIC address page '(BootVector(0x20220100) / 256) - 1 = 0x00000000'.
18:01:05 DEBUG - Getting the SBIC address      'SbicPage(0x00000000)    * 256      = 0x20220000'.
18:01:05 DEBUG - Invoking command: "C:/Microchip/Libero_SoC_v2022.2/Designer/bin64//fpgenprog.exe mss_boot_info --location c:\Microchip\hart-software-services\Default/bootmode3/fpgenprogProject --u_mss_bootmode 3 --u_mss_bootcfg 0000000000000000000000000000000020220000 --ucskx 36a32b90aed78a26fef24130e019dfbb819bb43a1d93446977fbc41a585996ca7fbfbbb917a90600db941a687a4c26b5 --ucsky ebcdbbfea25d1c2973492ff97a02bf59446229218d6ea8287dbe656073e3ada5b85fcedf968430a551da3509bc1d21bb --reset_sbic_version".
18:01:05 DEBUG - Finished with exit code: "0".
18:01:05 DEBUG - Getting the SBIC address page '(BootVector(0x20220100) / 256) - 1 = 0x00000000'.
18:01:05 DEBUG - Getting the SBIC address      'SbicPage(0x00000000)    * 256      = 0x20220000'.
18:01:05 DEBUG - Invoking command: "C:/Microchip/Libero_SoC_v2022.2/Designer/bin64//fpgenprog.exe envm_client --location c:\Microchip\hart-software-services\Default/bootmode3/fpgenprogProject --number_of_bytes 95440 --content_file_format intel-hex --content_file c:\Microchip\hart-software-services\Default/bootmode3/hss-envm-wrapper-bm3-p0.hex --start_page 0 --client_name bootmode3_0 --mem_file_base_address 20220000".
18:01:06 DEBUG - Finished with exit code: "0".
18:01:06 INFO  - Generating bitstream...
18:01:06 DEBUG - Invoking command: "C:/Microchip/Libero_SoC_v2022.2/Designer/bin64//fpgenprog.exe generate_bitstream --location c:\Microchip\hart-software-services\Default/bootmode3/fpgenprogProject".
18:01:21 DEBUG - Finished with exit code: "0".
18:01:21 INFO  - Programming the target...
18:01:21 DEBUG - Invoking command: "C:/Microchip/Libero_SoC_v2022.2/Designer/bin64//fpgenprog.exe run_action --location c:\Microchip\hart-software-services\Default/bootmode3/fpgenprogProject --action PROGRAM".
18:01:31 DEBUG - Finished with exit code: "0".
18:01:31 INFO  - Verifying the target...
18:01:31 DEBUG - Invoking command: "C:/Microchip/Libero_SoC_v2022.2/Designer/bin64//fpgenprog.exe run_action --location c:\Microchip\hart-software-services\Default/bootmode3/fpgenprogProject --action VERIFY".
18:01:38 DEBUG - Finished with exit code: "0".
18:01:38 INFO  - mpfsBootmodeProgrammer completed successfully.

Thanks for your inputs.

nearly-big-endian commented 1 year ago

Any update on this?

vfalanis commented 1 year ago

Hi @nearly-big-endian,

Thanks for providing the logs. There is known issue when using Boot Mode 3 (Factory Secure Boot) with applications targeting eNVM + L2-scratchpad. We are currently working on this. We'll provide a fix as soon as we complete our testing.

vfalanis commented 1 year ago

Hi @nearly-big-endian ,

This issue was fixed on 2023.02 release. Could you please update your icicle kit with the latest HSS?

Thanks

nearly-big-endian commented 1 year ago

Hello @vfalanis,

I am happy to report that HSS v0.99.35 now boots as expected in Secure Boot mode 3 on an IcicleKit where all versions of HSS / Yocto BSP / Ref. Design are aligned to the 2023.02 release.

Successfully tested with the HSS programmed through SoftConsole+fpgenprog, and through a LiberoSoC v2022.3 project (in a Boot Mode 3 eNVM Client).

So, thanks and congrats for the good work 👍

Just another question : is the fix relatively easy to back-port to HSS 2022.09 (matter of merging a couple of commits only) or is it too deeply nested with other changes to be a reasonable idea ?

Thanks !

vfalanis commented 1 year ago

Hi @nearly-big-endian

There are couple of updates (HAL version and some other commits related to cache flushing) required that are split into more than one commit, normally HAL updates involving several files are not that straightforward to back-port. We usually recommend to update to the latest release.

Is there any reason why you need 2022.09 specifically? We strongly recommend to update to latest release as there have been new features and improvements since then.

Thanks

vfalanis commented 1 year ago

Hey @nearly-big-endian, I'm closing this issue since it has been fixed in 2023.02 release, feel free to open it back up if you need any further clarification :)

polarfire-soc / hart-software-services

HSS not booting in Factory Secure Boot Mode 3 #53