search

robn / quiz

Fast edit-compile-test cycles for Linux+OpenZFS development
Mozilla Public License 2.0
23 stars 3 forks source link

readme

🧩 quiz: a tool for rapid OpenZFS development

quiz is a program to support fast edit-compile-test cycles for Linux kernel development, with a focus on OpenZFS. At its heart its a qemu-based microvm, stripped down to the essentials to setup and boot a VM and start a test program in under a second.

Motivation

My preferred programming style is highly interactive and exploratory, which requires a fast edit-compile-test cycle. But then I started working I work on OpenZFS, which is a big chunk of kernel code. Kernel code means developing on either real hardware or a VM, which tend to be awkward to get real code onto, and then, being kernel code, its pretty easy to crash, wedge or otherwise damage the kernel, requiring a reboot.

I wanted a way to work on OpenZFS just like I’d work on any other program - make a change, compile it, run it and see what happens. I happened to know that microVMs were getting to the point where booting in a fraction of a second was possible, so I decided to see if I could use one for this. quiz is what I came up with.

Note

This is still pretty rough; you need to be willing to get your hands dirty and customise things, and its still firmly in the experimental stage: I change how things work pretty often, as I understand better what I need.

That said, I'm using this to spin up tens and sometimes hundreds of VMs every day, doing Real Programming and furiously creating value for the shareholders. It does work and is safe to play with!

Just don't treat anything here as more than "highly experimental".

Demo

Simple boot to prompt:

quiz boot

Creating a ZFS pool over memory-backed devices:

quiz ZFS pool create

Running (part of) the ZFS test suite:

quiz ZFS test suite run

Setup

Install prerequisites

On a Debian 12 (bookworm) system you’ll need at least:

Build the kernel

Run quiz-prepare-kernel. This will grab kernel source, compile it with a very minimal config to support only what’s needed for the microvm, and install it into the system dir.

Build the root image

Run quiz-prepare-root. This will generate a barely-there Debian root image.

Build the work dir

Run quiz-prepare-work. This sets up a chunk of host filesystem that will be mapped into the VM, and is where the OpenZFS modules and related development programs can be stored.

Build OpenZFS

The configure and install commands are quite involved to get it to properly target the work dir. The quiz-build-zfs wrapper is there to help with this.

From an OpenZFS checkout:

$ ./autogen.sh
$ ~/quiz/quiz-build-zfs configure --enable-debug --enable-debuginfo # your configure options here
$ make -j5
$ ~/quiz/quiz-build-zfs make install

Run it

Simple boot to a shell:

$ ./quiz
[quiz] 20230909-23:35:00 starting microvm
[    0.318686] brd: module loaded
[    0.445527] loop: module loaded
[    0.451687] virtio_blk virtio2: [vda] 1608310 512-byte logical blocks (823 MB/785 MiB)
...
[    0.520927] Run /sbin/init as init process
[    0.640503] Using default interface naming scheme 'v252'.
[    0.851721] quiz: starting user program
[INFO  tini (1)] Spawned child process '/bin/bash' with pid '511'
bash: cannot set terminal process group (-1): Inappropriate ioctl for device
bash: no job control in this shell
root@quiz:/#

or run a program directly:

$ ./quiz uname -a
[quiz] 20230909-23:35:37 creating run script
[quiz] 20230909-23:35:37 starting microvm
[    0.317245] brd: module loaded
[    0.456128] loop: module loaded
[    0.463744] virtio_blk virtio2: [vda] 1608310 512-byte logical blocks (823 MB/785 MiB)
...
[    0.523497] Run /sbin/init as init process
[    0.714328] Using default interface naming scheme 'v252'.
[    0.913468] quiz: starting user program
[INFO  tini (1)] Spawned child process '/bin/sh' with pid '511'
+ uname -a
Linux quiz 5.10.170 #3 SMP Wed Jun 21 15:52:50 AEST 2023 x86_64 GNU/Linux
[INFO  tini (1)] Main child exited normally (with status '0')

Include some profiles:

$ ./quiz -p zfs zpool status
$ ./quiz -p zfs zpool status
[quiz] 20230909-23:41:05 including profile: zfs
[quiz] 20230909-23:41:05 creating run script
[quiz] 20230909-23:41:05 starting microvm
[    0.318037] brd: module loaded
[    0.444730] loop: module loaded
[    0.452162] virtio_blk virtio2: [vda] 1608310 512-byte logical blocks (823 MB/785 MiB)
...
[    0.886639] quiz: starting profile init: zfs
[    1.075146] spl: loading out-of-tree module taints kernel.
[    2.223443] zfs: module license 'CDDL' taints kernel.
[    2.223489] Disabling lock debugging due to kernel taint
[    3.145029] ZFS: Loaded module v2.2.99-62_g28fdf9f64 (DEBUG mode), ZFS pool version 5000, ZFS filesystem version 5
[    3.146129] quiz: starting user program
[INFO  tini (1)] Spawned child process '/bin/sh' with pid '526'
+ zpool status
no pools available
[INFO  tini (1)] Main child exited normally (with status '0')

Config

Some amount of config can be done by creating a quiz.cfg. Defaults are set in quiz-config (which loads quiz.cfg to override them). The core scripts and profiles are configured this way.

Profiles

Profiles are quiz's extension mechanism. They introduce additional setup commands, config files or even whole programs into the VM as its being generated. The idea is that you can just enable the bits you need as you need them, to keep the boot time as small as possible for any given task.

Profiles are enabled with the -p switch to quiz, or via the QUIZ_PROFILE config variable.

The following profiles exist:

Profiles are in the profile/ dir, each in their own named dir. There's three main hooks available:

Note that profiles currently have no way to influence the kernel config or the root filesystem, which is a problem if a profile needs something from it. I haven't decided what, if anything, to do about this yet.

Todo

More info