Default ZFS datasets are very limited

[bretton]

The default ZFS datasets in mfsbsd setup are very limited:

root@depenguintest:~ # zfs list
NAME             USED  AVAIL     REFER  MOUNTPOINT
zroot           1.05G  1.76T       24K  none
zroot/root      1.05G  1.76T      938M  /
zroot/root/tmp    25K  1.76T       25K  /tmp
zroot/root/var   133M  1.76T      133M  /var

By comparison a normal FreeBSD setup (13.0) would create something more like

zroot                                               53.7G   373G       96K  /zroot
zroot/ROOT                                          46.4G   373G       96K  none
zroot/ROOT/default                                  46.4G   373G     46.4G  /
zroot/tmp                                           2.50G   373G     2.50G  /tmp
zroot/usr                                           3.45G   373G       96K  /usr
zroot/usr/home                                      2.51G   373G     2.51G  /usr/home
zroot/usr/ports                                       96K   373G       96K  /usr/ports
zroot/usr/src                                        963M   373G      963M  /usr/src
zroot/var                                           9.98M   373G       96K  /var
zroot/var/audit                                       96K   373G       96K  /var/audit
zroot/var/crash                                       96K   373G       96K  /var/crash
zroot/var/log                                       4.52M   373G     4.52M  /var/log
zroot/var/mail                                      5.07M   373G     5.07M  /var/mail
zroot/var/tmp                                        112K   373G      112K  /var/tmp

with user, or other programs like pot, creating additional datasets.

I'd like to suggest an improvement here, but not sure what. Or draw more attention to the limited setup somehow.

[grembo]

@bretton What I usually do is boot into mfsBSD and then use normal bsdinstall to install the OS (I download the required tarballs before calling it, so that they’re available within tmpfs already). Works fine for me and always gives me a "standard" installation.

[bretton]

Does that still give root on zfs?

can do that and use installerconfig script like with packer

[grembo]

Does that still give root on zfs?

can do that and use installerconfig script like with packer

Of course, root on zfs has been in the standard installer for many many years, no problem (boot environments rely on it as well). Also allows you to configure encrypted swap etc

[bretton]

Lets set a cut off point? What is preferred?

• The script takes user to a point of manually running zfsinstall or bsdinstall. No more, no less.
• The script automates provisioning of ZFS datasets, network, users with a scripted bsdinstall
• The script automates provisioning of ZFS and encrypted swap, network, users with a scripted bsdinstall
• The script automates provisioning of ZFS and encrypted swap, network, users with a scripted bsdinstall and prepares for ansible configuration
• The script automates provisioning of ZFS and encrypted swap, network, users with a scripted bsdinstall and $(builds-a-cluster-node)

Limits

• The script works on hetzner
• The script works on platforms with a pxe linux rescue system
• The script works on all dedicated server platforms
• The script works on all cloud and dedicated server platforms

[click0]

Encrypted swap has been around since FreeBSD 13.0 Still need to provide support for installation on UEFI and GELI.

[grembo]

Encrypted swap has been around since FreeBSD 13.0

I think bsdinstall supports it since 2013 (with zfs). The OS itself at least since 8.0 I think.

Still need to provide support for installation on UEFI and GELI.

bsdinstall can do encrypted root also for quite some time. I think it supports UEFI+GELI without a separate boot partition since 2018 (with zfs). This would require console access, so more useful for laptops. What probably makes more sense in a DC is to either add an additional data partition or use zfs encryption (available since 13.0)

Unless there is a better solution to enter credentials for zroot I don’t know about yet :)

Not sure about UFS capabilities in current bsdinstall.

@bretton I think we should draw the line at the point when the user can decide how to install the system. To me, the most important thing about this image is to be able to rescue a server that was already configured (so not so much about fresh installation, where it obviously also saves time and money and which is certainly also very important). We could of course provide additional useful scripts inside our custom mfsBSD image (as part of the builder repo, from where people could also fetch them). Could even get its own repo or become a package over time. But for this script itself, limiting it on getting a machine with a running mfsBSD image seems reasonable. Everything else could happen in the custom image (and if users decide to use their own/a vanilla mfsBSD image, we won’t have to bother them).

Ideally, this should work on all providers where a Linux memory fs (Hetzner rescue-like) is offered. Provider specific scripts etc could - again - be provided in our custom mfsBSD image or by others who happen to use these specific providers.

For providers where only an initial Linux installation works and no rescue system exists (so our approach won’t really work), using our script is pointless and people will use standard mfsBSD or someone else’s streamlined effort (potentially different approaches for different providers).

At least that’s what I think ;)

[bretton]

I'm happy with a tool that goes from a shell one-liner to a prompt where you can choose to:

• rescue-this-host
• wipe-this-host
• clone-host-disks-offsite
• zfsinstall
• curses-based bsdinstall
• scripted bsdinstall (pass in installerconfig)
• scripted bsdinstall (multiple preset flavours)
• let automation tools like ansible take over

and the approach of packing into the mfsbsd image created.

I'd like to see something which can be used on many provider platforms.

[bretton]

Still need to provide support for installation on UEFI and GELI.

it appears this is already enabled using the options:

MYBIOS="/usr/share/ovmf/OVMF.fd"
...
-bios "${MYBIOS}"

I can confirm the TianoCore boot loader is used, as covered in this link

Was something else required? I fought with qemu and bios/uefi on some other stuff and see now it's not nearly as complicated as the howtos make it. The above is all that's needed with current qemu versions.

A test on a system with more than 2TB can be done when sponsorships allow, or someone does it for their servers.

[click0]

I'd like to see something which can be used on many provider platforms.

I am using a modified go9.sh script. It does not support UEFI and GELI. There is a role for ansible that runs a script inside tmux inside MfsBSD mini. There is a separate script install_mfsbsd_iso.sh which, after rebooting the Linux host, launches iso MfsBSD with the necessary network settings (if necessary), without modifying the MfsBSD image. It also has a role for Ansible.

[grembo]

I'd like to see something which can be used on many provider platforms.

I am using a modified go9.sh script. It does not support UEFI and GELI. There is a role for ansible that runs a script inside tmux inside MfsBSD mini. There is a separate script install_mfsbsd_iso.sh which, after rebooting the Linux host, launches iso MfsBSD with the necessary network settings (if necessary), without modifying the MfsBSD image. It also has a role for Ansible.

Mind sharing those scripts/roles?

[click0]

Mind sharing those scripts/roles?

I need some time to translate the description into English and comply with Ansible standards. Today or tomorrow I will post scripts and roles.

[click0]

Mind sharing those scripts/roles?

I need some time to translate the description into English and comply with Ansible standards. Today or tomorrow I will post scripts and roles.

I posted a project with scripts. Please do not write a long wish list. Projects with Ansible roles will come later.

[click0]

Mind sharing those scripts/roles?

I need some time to translate the description into English and comply with Ansible standards. Today or tomorrow I will post scripts and roles.

I posted a project with scripts. Please do not write a long wish list. Projects with Ansible roles will come later.

I published Ansible role:

1. ansible-mfsbsd-install-via-linux-lite
2. ansible-freebsd-install-on-zfs-lite

[bretton]

closing with solution for scripted bsdinstall or run manually