UEFI SB release - Githubissues

TomaszAIR commented 7 months ago

This Pull Request gather changes made in OSFV after working on Secure Boot Test Suite for MPL PIP platform that uses AMI BIOS. Following modifications are introduced here:

Added new UEFI Secure Boot test cases: SBO009 - SBO015
Split BIOS menu related keywords into Dasharo/AMI/common
- changes in multiple files to drop Tianocore from names
- KW names duplicates between Dasharo/AMI so the test case can use the same KW names but depending on used platform configuration, Dasharo or AMI definition of given KW will be used
Add script to generate images used for UEFI Secure Boot tests
Introduce ansible usage as a part of Suite Setup, to perform additional actions on booted OSes
- adds readme entry about usage
Added couple new variables to platform-configs

What could be changed

Some tests depends on booting DTS with UEFI SB integrated, need a way how to send its image if it is not on PiKVM
Ansible command and hosts configuration could be handled better
- now for every new device we need to add new hosts entry
- ansible is run for every host form hosts file, failures on not available hosts are skipped

TomaszAIR commented 5 months ago

Results on QEMU after latest rebase on develop below. QEMU started with DRIVE_PATH=./qemu-data/dts-base-sb-image-genericx86-64.wic ./scripts/ci/qemu-run.sh graphic os command, dts-base-sb-image-genericx86-64.wic is decompressed DTS image built by using https://github.com/Dasharo/meta-dts/blob/main/kas-uefi-sb.yml.

λ robot -b file.txt -L TRACE -v ansible_config:yes -v rte_ip:127.0.0.1 -v snipeit:no -v config:qemu dasharo-security/secure-boot.robot
==============================================================================
Secure-Boot
==============================================================================
SBO001.001 Check Secure Boot default state (firmware) :: This test... | PASS |
------------------------------------------------------------------------------
SBO002.001 UEFI Secure Boot (Ubuntu 22.04) :: This test verifies t... | PASS |
------------------------------------------------------------------------------
SBO002.002 UEFI Secure Boot (Windows 11) :: This test verifies tha... | SKIP |
SBO002.002 not supported
------------------------------------------------------------------------------
SBO003.001 Attempt to boot file with the correct key from Shell (f... | PASS |
------------------------------------------------------------------------------
SBO004.001 Attempt to boot file without the key from Shell (firmwa... | PASS |
------------------------------------------------------------------------------
SBO005.001 Attempt to boot file with the wrong-signed key from She... | PASS |
------------------------------------------------------------------------------
SBO006.001 Reset Secure Boot Keys option availability (firmware) :... | PASS |
------------------------------------------------------------------------------
SBO007.001 Attempt to boot the file after restoring keys to defaul... | PASS |
------------------------------------------------------------------------------
SBO008.001 Attempt to enroll the key in the incorrect format (firm... | PASS |
------------------------------------------------------------------------------
SBO009.001 Attempt to boot file signed for intermediate certificat... | PASS |
------------------------------------------------------------------------------
SBO010.001 Check support for rsa2k signed certificates :: PEM gene... | PASS |
------------------------------------------------------------------------------
SBO010.002 Check support for rsa3k signed certificates :: PEM gene... | PASS |
------------------------------------------------------------------------------
SBO010.003 Check support for rsa4k signed certificates :: PEM gene... | PASS |
------------------------------------------------------------------------------
SBO010.004 Check support for ecdsa256 signed certificates :: PEM g... | FAIL |
'
Command Error Status: Access Denied
FS0:\>' does not contain 'hello, world!'
------------------------------------------------------------------------------
SBO010.005 Check support for ecdsa384 signed certificates :: PEM g... | FAIL |
'
Command Error Status: Access Denied
FS0:\>' does not contain 'hello, world!'
------------------------------------------------------------------------------
SBO010.006 Check support for ecdsa521 signed certificates :: PEM g... | FAIL |
'
Command Error Status: Access Denied
FS0:\>' does not contain 'hello, world!'
------------------------------------------------------------------------------
SBO011.001 Attempt to enroll expired certificate and boot signed i... | FAIL |
'
Hello, world!
FS0:\>' does not contain any of 'Access Denied'
------------------------------------------------------------------------------
SBO012.001 Boot OS Signed And Enrolled From Inside System (Ubuntu ... | PASS |
------------------------------------------------------------------------------
SBO013.001 Check automatic certificate provisioning :: This test v... | PASS |
------------------------------------------------------------------------------
SBO013.002 Check automatic certificate provisioning KEK certificat... | PASS |
------------------------------------------------------------------------------
SBO014.001 Enroll certificates using sbctl :: This test erases Sec... | PASS |
------------------------------------------------------------------------------
SBO015.001 Attempt to enroll the key in the incorrect format (OS) ... | PASS |
------------------------------------------------------------------------------
Secure-Boot                                                           | FAIL |
22 tests, 17 passed, 4 failed, 1 skipped
==============================================================================
Debug:   /home/tzyjewski/files/open-source-firmware-validation/file.txt
Output:  /home/tzyjewski/files/open-source-firmware-validation/output.xml
Log:     /home/tzyjewski/files/open-source-firmware-validation/log.html
Report:  /home/tzyjewski/files/open-source-firmware-validation/report.html

ecdsa tests fails as BIOS does not accepts ecdsa-based certs

test with expired cert fails, as BIOS can still execute efi file signed with expired cert

TomaszAIR commented 4 months ago

Latest results on msi pro z690-a ddr5 results.tar.gz

Sometimes suite teardown fails (or any other test) because test e.g. goes into HDD Security Configuration instead of Secure Boot Configuration. Looks like it sometimes slip one step. results-teardown.tar.gz

TomaszAIR commented 3 months ago

@macpijan I responded in all threads. I think that @m-iwanicki @PLangowski @DaniilKl might help if there will be additional questions.

Dasharo / open-source-firmware-validation

UEFI SB release #187