Proposal: base image to act as host for project container

[cmacq2]

With the R.Pi2 kit we have a problem/barrier to collaboration in that:

• All of us will need to climb the same first hill: getting something to boot.
• Many of us will not want to use the SD card as a read-writable rootfs, but rather offload most if not all of the IO to the shipped WD disks.
• These disks may not come preformatted, or at least it would be nice if the rootfs could set this up automatically in order for consumers to be able to plug in 'blank' disks and have it come up by itself.
• A lot of us will probably (at least initially) diverge in 'direction' of what the rootfs should contain.

I think that the following approach will permit us to solve issues one thru three in one go, and also permit each of us full flexibility w.r.t. to item 4:

• Let's create a simple, minimal base image. I propose we use Debian for it -- there's good tools/infra for building custom images and the end-user still has a hope of updating/admin'ing it manually (in particular I think that compiling things from source on the RPi is a recipe for disaster because most end-users won't have the skills or patience to sit through all that -- so arch is probably cool but not very practical).
• The base image contains a few container tools. With Debian the package systemd, systemd-sysv, and systemd the image ought to be capable of booting LXC containers in a few formats, notably raw disk images in QCOW and docker formats. (Through machinectl).
• The image should ship with auto-discovery and auto-setup .service/.mount/etc files to detect the state of the /dev/sd* WD disks and partition these automatically.
• The image should ship with automatic network setup for IPv4 and IPv6 DHCP.
• I propose we use a GPT partitioning scheme to future proof it for larger disks and other considerations. We can use sgdisk to do the partitioning automatically.
• The image should set up partitions such that /var, /srv and /home point to the WD disks (array).
• (It may be wortwhile to investigate automatic RAID 0/RAID 1 or JBOD setup, using LVM.)
• The image may contain either an embedded container image or ship something to automatically build or pull a container image off the WWW/repos. This container should be given some well known default name so we can refer to it safely from integration bits, e.g. to check if the first-run set up has completed successfully so we don't redo first-run on reboot. I propose to use the simplest, most obvious name: 'default'. This is to be used with machinectl pull-* or machinectl import commands.
• The image should ship with .nspawn, *.sockets set up to point into the container and declare deps on having the right mount points available. http://www.freedesktop.org/software/systemd/man/systemd.nspawn.html
• Possibly an integration package providing some convenience functions and a first-run service binaries to set up things after partitioning disks.
• Possibly a cockpit/ cockpit-ws packages to provide remote admin interface for the power user/sys admin.
• Investigate support for a seed file to let systemd seed /dev/urandom. Alternatively only generate keys after formatting and building a container so there's some entropy from downloads, processor activity and disk I/O in the system.

[cmacq2]

Overview

Status

I've done some experimental work towards getting this base image thing done, based on brickstrap. You can build a rootfs which should boot, but I have not yet been able to test this. (I still lack a R.Pi2 to play with). The image needs some refining and trimming around the edges because I basically grabbed an existing board config and did some obvious changes based on a cursory review, like disabling most all packages that it shipped with by default.

For example the image lacks a entropy seed file, ifupdown config.

Also, the automatic provisioning of the WD disks is still at TODO stage.

What is this brickstrap thing?

Brickstrap is a nice tool to automate the creation of rootfs images as a normal user without requiring elevated privileges during the actual build. It's the evolution of various thoughts and work done in the spirit expressed in this old blog post. You may find it worth a read to get an overview of the "how things should work" concept behind it, and also the "why things should be done this way" aspect.

Highlights/features

• Build a complete bootable image, without touching any file on disk as actual root/sudo.
• Use the package manager of the distro to your advantage: you can have multiple repositories and get all the blessings of apt. The image uses APT pinning to make sure that the image/OS will track the upstream Debian repositories as much as possible by default except for Raspberry Pi specific things like kernel and firmware.
• Get a much more recent kernel than stock Raspbian, thanks to the efforts of the EV3DEV project (who are also the folks behind the upstream brickstrap project). (Kernel 4.1 IIRC, which is only one or two point releases behind Debian unstable or so. To me that looks much better than 3.something Raspbian seems stuck with.)
• The image uses pwgen to generate a non-guessable password for user accounts instead of fixed defaults. The actual password is currently printed in the output of brickstrap just before exiting. This reporting mechanism could also be used to print fingerprints of SSH keys if any are to be embedded by default. (Which I'm not sure on.)

Where do I get this hot new stuff?

The work I did is currently divided into two parts:

1. An experimental board configuration/setup for the RPi2 based off the upstream RPi2-with-Jessie to define the base image. This works lives the rpi2-owncloud-wd-minimal-base branch here. It's very much not done yet but can be built using brickstrap.
2. Some simple, generically useful improvements to the frontend script (brickstrap.sh itself). This work has been upstreamed. More ideas for improvements are being discussed upstream as well.

Tangentially related, I sent a patch to QEMU mailing lists to update it with a bunch of missing syscalls on arm, aarch64 and mips. (Notably get_random()).

How do I use this?

Build & develop

To build & develop the image further:

• Clone this repository: git clone git@github.com:cmacq2/brickstrap.git
• Create a tempory work branch which will be used to merge both feature branches into: git checkout -b temp-build-branch
• Merge board directory branch: git merge rpi2-owncloud-wd-minimal-base
• Make an empty build directory somewhere else and run brickstrap: mkdir -p ../tmp-builddir/out
• Run brickstrap: ./brickstrap.sh -f -b rpi2-wd-minimal -d ../tmp-builddir/out
• Use the rpi2-owncloud-wd-minimal-base branch to develop the image further, merge into your temp branch for building/testing purposes, and rebuild.
• The board configuration is located in rpi2-wd-minimal

Required tools/packages

Apart from brickstrap, the following tools are required:

• multistrap (used by brickstrap)
• uidmap (used by user-unshare.pl to setup private user namespace to act like a chroot jail)
• libguestfs-tools (to create the image and partition it)
• qemu-system-arm, qemu-user-static, binfmt-support (to run the armhf binaries as part of package configuration/shell commands executed in the private user namespace)
• apt/dpkg etc. (or rather their Perl libs used by multistrap for package management purposes).
• pwgen (used for generating decent-ish passwords for user accounts by default)

[oparoz]

Added to the wiki https://github.com/owncloud/pi-image/wiki/Alternative-images