Automated test

[smoser]

We would like to have some way to provide automated "integration" test inside the stubby tree. I added some unit-test like functionality, but actually booting a UEFI system with a signed kernel.efi is ultimately what we need to test.

Goals

• Test boot of efi application (kernel.efi) that was built using the just-built stubby.efi and assembled via stubby-smash.2.sh.
• No need to test a shim, sign the kernel.efi with a key that is in the firmware's database.
• Verify boot of built-in command line
• Verify boot of cli provided command line for terms in the allowed_list
• Verify failure of cli provided command line for terms not in the allowed list.

Implementation suggestion

• Use github workflows and execute in an ubuntu focal environment. (see the build workflow for an example).
• install qemu-system-x86 and ovmf and use qemu-system-x86_64 to boot a system.
• get a kernel and initramfs from cirros as described in a gist I wrote recently. Its probably best to copy that content to this repo.

• Then, iterate over the following, varying kernel command lines

  • smash kernel, initramfs, cmdline into kernel.efi

  • sign kernel.efi with the snakeoil keys that are provided by ovmf package (doc

  • create a vfat/esp image with that kernel inside and a startup.nsh script.

    • Another gist contains gen-esp that allows a single command for making an esp image. It has usage like:

      $ ./tools/gen-esp create esp.img \ kernel.efi:kernel.efi \ startup.nsh:startup.nsh

      creating image 128MB in esp.img EFI/BOOT/STARTUP.NSH -> EFI/BOOT/STARTUP.NSH EFI/BOOT/KERNEL.EFI -> EFI/BOOT/KERNEL.EFI

  • start the VM, with the built esp.img as a disk. Log its console output. Expect it to shutdown itself or kill it and fail after ~2 minutes.

    • Note that starting with no attached network devices will probably boot faster as it wont' attempt to pxe.

  • inspect console output to determine what happened (was the boot allowed, did it boot correctly ..)

  • report PASS or FAIL

[smoser]

Here is an initial test matrix.

• 'empty' means empty, not the liter string. 'none' means no arguments provided to kernel.efi
• WW: in non-secureboot mode, a warning 'Custom kernel would be rejected in secure mode' should be output. WW should only appear in secureboot=false. Anytime there is a WW in secureboot=false, the equivalent secureboot=true should FAIL.

builtin	secureboot	provided	behavior	message	notes
empty	true	empty	boot	cmdline: empty	empty cmdline is fine
empty	false	empty	boot	cmdline: empty	empty cmdline is fine
empty	true	key=val	FAIL	token not allowed	illegal token in SB
empty	false	key=val	boot	WW cmdline: key=val	illegal token is allowed in ISB
empty	true	root=atomix	boot	cmdline: root=atomix	legal token in SB
empty	false	root=atomix	boot	cmdline: root=atomix	legal token is in ISB
empty	true	root=atomix verbose	boot	cmdline: root=atomix verbose	two legal tokens SB
empty	false	root=atomix verbose	boot	cmdline: root=atomix verbose	two legal tokens ISB
empty	true	root=atomix verbose bad	FAIL	token not allowed	two legal one illegal SB
empty	false	root=atomix verbose bad	boot	WW cmdline: root=atomix verbose bad	two legal, one illegal ISB
key=val	true	none	boot	cmdline: key=val	illegal token builtin SB
key=val	false	none	boot	cmdline: key=val	illegal token builtin ISB
root=atomix	true	key=val	FAIL	Custom kernel command line rejected	illegal token SB
root=atomix	false	key=val	boot	WW cmdline: key=val	illegal token ISB
root=atomix	true	verbose	FAIL	Custom kernel command line rejected	override SB
root=atomix	false	verbose	boot	WW cmdline: verbose	override ISB

[berlin4apk]

Can we please also add a CI test for PXE boot?

[smoser]

I'm working on this and it is close to complete (sorry, no PXE boot). Its on my feature/c-i-boot-test branch.

[smoser]

Marking this as fixed in #21.