Feature request: how might Debian/Ubuntu ZFS-on-root guides become scriptable?

[Intensity]

The guides are great and informative, especially for system administrators. Having the commands coupled with explanations lets the administrator or developer create a ZFS-on-Linux root using the suggested configuration, and with some additional knowledge this can be tweaked to the preferences and needs of the system.

That said, I'm wondering in what ways these guides might be augmented to become a declarative specification such that a script automates the process of creating an image or deploying onto a system.

What about leveraging packer? https://www.packer.io - or offering declarative functionality similar to NixOps? https://nixos.org/nixops/

I'd be interested in automatically deploying to new bare metal hosts (including a Hetzner machine), automatically generating Virtualbox and VMware Workstation/ESX (OVA) images, AMI and Google Compute Engine images, and so on. In particular, with these different artifacts, getting the device specification right and making it portable and repeatable is important - I'd like to ensure that the ZFS pool is referred to by the right entry in /dev/disk/ to ensure transparent booting into the image even if the VM is generated and deployed from an OVA. Thus far, that particular piece hasn't worked very well - since the disk ID might change and might need a manual override to some canonical value in the metadata.

Also, some way to create an encrypted ZFS pool would be helpful. Both Gentoo and NixOS have prior efforts underway to support ZFS-on-root. I'm interested in doing the same for Debian (7), and in particular I'd like to have the input tweakable so that the most relevant pieces of the result can be changed quickly. That's subject to interpretation, so I'd invite feedback (maybe there's a way to specify the disk configuration or the resultant image size and so on). Alternatively I think it would be helpful to piggyback upon related and similar efforts - in what other ways are Debian/Ubuntu systems automatically generated from an initial specification, besides a preseed file (which appears to be less flexible)?

Further enhancements might include how to "migrate" the root filesystem of one pre-existing running system and deploy it onto an image of another type (for example, suppose there's a system running on bare metal; instead of the debootstrap phase, the system could be migrated to, say, a representative OVA that carries the same information content - via a zfs send for instance). This would need to involve a clear separation of the particular low-level ZFS-on-root configuration from the rest of the system's booting (that is to say, GRUB boot configuration, /etc/fstab, /etc/modules and other information in /boot/ and /lib/modules/ might need to be particular to the destination image, but still the rest of the system could be copied over).

[azeemism]

@Intensity the following may be helpful https://github.com/zfsonlinux/pkg-zfs/issues/200#issuecomment-206433104

[FransUrbo]

If you're really, really brave (and have a lot of luck), you might be able to use the mini ISO I created a year or so ago - http://archive.zfsonlinux.org/debian/mini.iso

Haven't tried it since I created it (and it worked then), so it might not even work.

[FransUrbo]

Btw, just for the record: I was the first to provide ZFS on root for the Debian GNU/Linux distribution! And it have been working for several years in my packages. The other dists is still trying to catch up on that work.

[FransUrbo]

Actually, it seems like the ISO still works! It installs Wheezy though and I have yet to try to upgrade to Jessie.

[FransUrbo]

Just tested the upgrade from Wheezy to Jessie and that worked as well. Unfortunately, out of my control, I can't offer automatic upgrades that way :(.

But doing it manually isn't that difficult.

Snapshot fs

Before doing an upgrade, I suggest doing a snapshot of the filesystem:

zfs snapshot -r "$(mount | grep ' on / type zfs' | sed 's, on .*,,')@pre-jessie"

This so you can go back and try the upgrade again if it fails by booting with

root=ZFS=<your root fs name>@pre-jessie

on the Grub command line.

Change the sources list files:

sed -i 's,wheezy,jessie,g' /etc/apt/sources.list
sed -i 's,wheezy,jessie,g' /etc/apt/sources.list.d/zfsonlinux.list

Upgrade base system

run apt-get update && apt-get dist-upgrade and that finished successfully, do NOT reboot!

Upgrade SPL & ZFS

... but instead continue upgrading the SPL and ZFS packages:

splv=$(apt-cache show spl | grep ^Version: | tail -n1 | sed 's,.*: ,,'); \
zfsv=$(apt-cache show zfsutils | grep ^Version: | tail -n1 | sed 's,.*: ,,'); \
apt-get -d install spl=$splv spl-dkms=$splv libnvpair1=$zfsv libuutil1=$zfsv \
libzfs2=$zfsv libzpool2=$zfsv zfs-dkms=$zfsv zfs-initramfs=$zfsv zfsutils=$zfsv

!! PAY ATTENTION !!

Make sure to pay attention on the Building initial module for ... parts to make sure the module(s) is built correctly!

PS: apt-get will tell you that it will downgrade the package. This is fictional only and can be ignore. Just indicate that's what you want.

Setting up spl (0.6.5.6-3) ...
Setting up spl-dkms (0.6.5.6-3) ...
Loading new spl-0.6.5.6 DKMS files...
Building only for 3.2.0-4-amd64
Building initial module for 3.2.0-4-amd64
Done.
[....]

[not sure why I'm not getting the new 3.16 kernel though]

[zidarko]

@FransUrbo:

I followed your instructions and managed to install & upgrade to Jessie. I was not neccessary to do "Upgrade SPL & ZFS". apt-get warned I'd be downgrading. The system still booted just fine.

Thanks.

[FransUrbo]

Right. But the downgrade is fictional only. I'll update my comment.