search

linuxboot / heads

A minimal Linux that runs as a coreboot or LinuxBoot ROM payload to provide a secure, flexible boot environment for laptops, workstations and servers.
https://osresearch.net/
GNU General Public License v2.0
1.42k stars 187 forks source link

Unit tests #987

Open rugk opened 3 years ago

rugk commented 3 years ago

I had a quick look at the code base and tried to find some unit tests for the biggest part of the used language (shell/bash). I could not find any, so I'm wondering whether you have any? Also looked at the CI script, but it just seems to build a whole lot of stuff.

If not, I'd suggest to add some, even starting with only some basic tests. For a security-relevant product at least some basic things may be tested.

To do this, I personally had a good experience with https://github.com/kward/shunit2/. Maybe this would be a good idea?

tlaurion commented 3 years ago

@rugk : That would be lovely!

A small historical background would be important:

  1. The Qemu platform (also built from CI) does not take advantage of the TPM. This is explicit in board config
  2. The Qemu run script (make BOARD=qemu-coreboot-fbwhiptail run) in board config doesn't take advantage of swtpm (same issue as above) while USB support is not in either (not passed at run either)
  3. Platforms still relying on coreboot 4.8.1 relies on this patch to implement measured boot
  4. Newer and next versions of board configs are based on coreboot measured boot implementations. Heads only take the measurements of the TPM to generate tpmtotp, and reuse the same secret for HOTP challenge over Librem Key/Nitrokey Pro/Nitrokey Storage.

Consequently, i'm not sure of what unit test we could really do here. Once qemu+swtpm+usb support is in, openqa could be applied to poke arount/make sure things are not breaking from one commit to the other on a functionality level. Otherwise, Heads is built in such a way that going into recovery is wiping the secret and extending PCRs so that measurements are made invalid.

But suggestions/PR welcome, as usual! What unit tests you would find useful here?

Thrilleratplay commented 3 years ago

@rugk I thought the same but the issue is far more complex than it seems. It was not able to have the scripts shellcheck compliant without creating errors. There are global variables that are hard to track down, different coding styles, possibly dead code and functions do parse variables in different ways. As there are so many platforms that no one developer has access to I did not know how to approach that task by starting with unit tests.

I tried to create a list of steps to take to get to a point where unit testing would be viable in #885. I remember running into an issue with mocking in shunit2 I could not figure out and have used shellspec on other projects. The unit test framework is ultimately up to personal tastes.

I have wanted to get back to that but I fall into on rabbit hole after another with other projects and have yet to circle back.

rugk commented 3 years ago

https://shellspec.info/ looks great, thanks, did not know that. Anyway, that's off-topic. :smile:

Thrilleratplay commented 3 years ago

@rugk Not really off topic as this is regarding unit testing the shell scripts in Heads. I suggested it as I had more success with shellspec than shunit2 and bats. Presumably as I am familiar with BDD style unit testing frameworks, but not everyone is. If you want to see some examples, here is where I used it before