Debian 12 "bookworm" - Githubissues

darkbasic commented 1 year ago

@rlaager is it safe to follow the Debian 11 Bullseye Root on ZFS HOWTO for Debian 12? Is there any change planned for Debian 12 "bookworm"?

rlaager commented 1 year ago

It's probably safe. I haven't had a chance to test it. I've been too busy with other things in life. Sorry!

I do intend on testing it soon and cutting a new version of the guide.

jim-perkins commented 1 year ago

Thanks for the very good tutorial. Just went thru it on bookworm. Here are my notes. If you would like pull requests just let me know and if you would like it all in one request or seperate request for each change. thanks again

Step 1: Prepare The Install Environment

Boot the Debian GNU/Linux Live CD. debian-live-12.0.0-amd64-standard.iso note- this is the live install image used

I actually did step 3 here, as my ipmi interface isnt cut and paste

Setup and update the repositories: deb http://deb.debian.org/debian/ bookworm main non-free-firmware contrib deb-src http://deb.debian.org/debian/ bookworm main non-free-firmware contrib deb [trusted=yes] file:/run/live/medium bookworm main non-free-firmware note- I had read in the release notes about the new non-free-firmware section

4 . Disable automounting: -bash: gsettings: command not found note- probably related to the image I used in step 1

Step 2: Disk Formatting

Create the boot pool: ${DISK}-part3 note – when using a kvm virt machine and /dev/vdx the partitions are /dev/vdxy although this isnt related to bullseye vs bookworm

Step 3: System Installation

Install the minimal system: debootstrap bookworm /mnt note- updated to bookworm

Step 4: System Configuration

Configure the package sources:

deb http://deb.debian.org/debian bookworm main contrib non-free-firmware deb-src http://deb.debian.org/debian bookworm main contrib non-free-firmware

deb http://deb.debian.org/debian-security bookworm-security main contrib non-free-firmware deb-src http://deb.debian.org/debian-security bookworm-security main contrib non-free-firmware

deb http://deb.debian.org/debian bookworm-updates main contrib non-free-firmware deb-src http://deb.debian.org/debian bookworm-updates main contrib non-free-firmware

note- updated from bullseye to bookworm and added non-free-firmware

Configure a basic system environment: ln -s /proc/self/mounts /etc/mtab ln: failed to create symbolic link '/etc/mtab': File exists note – got this error but things seemed to work fine
Install an NTP service to synchronize time timedatectl System has not been booted with systemd as init system (PID 1). Can't operate. Failed to connect to bus: Host is down note-got this error but things seemed to work fine

15: Optional: For ZFS native encryption or LUKS, configure Dropbear for remote unlocking: the location for configuration of dropbear has moved to Note: Earlier versions of dropbear-initramfs stored configuration files under /etc/dropbear-initramfs. In the version I'm using (v2020.81) this has moved to /etc/dropbear/initramfs and is what I'll be referencing. Substitute the path to suit your installed version.

rlaager commented 1 year ago

Awesome! If you are willing to help with this effort, I'll certainly take that help.

First off, see this: https://openzfs.github.io/openzfs-docs/Getting%20Started/Debian/Debian%20Bullseye%20Root%20on%20ZFS.html#contributing

The changes will always include:

Copy the Bullseye file to a version for Bookworm.
Link it from the index page.
Add the "Newer release available" section to Bullseye. (See Buster for an example.) This will link to the Bookworm instructions.
Change all instances of Bullseye to Bookworm, e.g. in sources files.
Update the ISO link.

Then it comes down to whatever actually changed. You listed some:

gsettings... That's really bizarre. gsettings is in libglib2.0-bin. That really doesn't exist in the live environment? Is it a GNOME environment? If it's a GNOME environment and gsettings really is not installed, then I suppose we need to add a sudo apt install libglib2.0-bin before that line. (I can't imagine automounting is disabled by default in the live environment, so we still need to be able to change that setting.)
/etc/mtab... If it exists by default, we can just drop that step.
timedatectl... does it work outside of the chroot (i.e. if you just boot a live environment, does it work)?
dropbear... if the path has changed, definitely update it.

One PR is fine.

Even one commit is fine for something like this. If you really wanted to do multiple commits, I think the logical way to split them up would be the one that copies the instructions and makes the "always include" changes from above and subsequent commits for the actual changes.

jim-perkins commented 1 year ago

Awesome! If you are willing to help with this effort, I'll certainly take that help.

Work in progress here. I will make a pull request when it is complete https://github.com/jim-perkins/openzfs-docs/tree/Debian-Bookworm

I am a baby using git so the commit history may be funky but it will all be good in the end. Probably finish in the next day or 2.

The changes will always include:
2. Link it from the index page.
Happened automatically
5. Update the ISO link.
It already linked to 'current' so we're good here https://cdimage.debian.org/mirror/cdimage/release/current-live/amd64/iso-hybrid/

Then it comes down to whatever actually changed. You listed some:
* gsettings... That's really bizarre. gsettings is in libglib2.0-bin. That really doesn't exist in the live environment? Is it a GNOME environment? If it's a GNOME environment and gsettings really is not installed, then I suppose we need to add a `sudo apt install libglib2.0-bin` before that line. (I can't imagine automounting is disabled by default in the live environment, so we still need to be able to change that setting.)
Yea I dfdnt read the instructions well and downloaded the 'standard' iso which doesnt include X11/GUI . the 'Gnome' version has gsettings. We can let this go . The tutorial doesnt need Xll though, worked fine for me from the command line.
* /etc/mtab... If it exists by default, we can just drop that step.
yes it existed in the chroot. so I will drop it
* timedatectl... does it work outside of the chroot (i.e. if you just boot a live environment, does it work)?
yes it was only in the chroot that it gave an error. Not sure how to handle this so will let it be for now
* dropbear... if the path has changed, definitely update it.
Yes the path has changed . This one took me a bit to figure out.

-- a question -- What about the non-free-firmware component ? I can add it to the sources.list or leave it out. I added it to all and didnt get any errors.
https://wiki.debian.org/Firmware#Debian_12_.28bookworm.29_and_later

darkbasic commented 1 year ago

https://github.com/openzfs/openzfs-docs/pull/433 Isn't this something we should also take into consideration for bookworm?

rlaager commented 1 year ago

Don't worry about the git history. If you end up with a million fix commits, that's fine. I can squash together for the merge.

I generally use the GUI version of the live environment. I figure that's more user friendly. Even in the case of people that "know what they are doing", it can be nice (e.g. having a web browser to look at the instructions and copy-and-paste).

We should include non-free-firmware, as Debian does that by default. People that don't want it can remove it.

Yes, we should pull in the changes from PR #433.

jim-perkins commented 1 year ago

Made a PR. Never tested a EUFI installation. Tested a kvm virt install and a physical install , both with 2 disks in a mirror situation, Had to -f the zfs creates on reused zfs disks, even after wiping per the instructions. but maybe that was just me. DIdnt put that in the changes.

It seems we are aware that the zfs rpool fails to export coming out of the chroot and the first install always requires a zpool import -f rpool . Never tested if that was possible via the dropbear initramfs ssh .

Let me know if any follow up is needed. thanks again for all the work with open zfs

rlaager commented 1 year ago

It seems we are aware that the zfs rpool fails to export coming out of the chroot and the first install always requires a zpool import -f rpool

Indeed. My plan (which I've just not had time to get to) is to add notes on this to the instructions. I just can't find any way to get it to unmount cleanly!

darkbasic commented 1 year ago

What's your experience with zfsbootmenu? I need the ability to boot into old snapshots and eventually rollback so it looks like a good alternative to grub. I used to use zsys in the past but it's an unmaintained nightmare nowadays.

rlaager commented 1 year ago

I haven't personally used it. I believe Jim (author of syncoid, etc.) recommends it. My "problem" with it is that getting it into the distro would be a pain, because to my understanding, it's basically another build of Linux.

That said, while I try to follow stock distro practices, it'd probably be fine to have this an optional add-on.

darkbasic commented 1 year ago

My "problem" with it is that getting it into the distro would be a pain, because to my understanding, it's basically another build of Linux.

There are binaries available, it's basically a curl command away.

colmbuckley commented 1 year ago

Step 2: Disk Formatting 4. Create the boot pool: ${DISK}-part3 note – when using a kvm virt machine and /dev/vdx the partitions are /dev/vdxy although this isnt related to bullseye vs bookworm

Note that the creation of the bpool pool can be simplified using -o compatibility=grub2 rather than -d and the long list of features - ie:

root@debian:~# zpool create \
    -o ashift=12 \
    -o autotrim=on \
    -o cachefile=/etc/zfs/zpool.cache \
    -o compatibility=grub2 \
    -O devices=off \
    -O acltype=posixacl -O xattr=sa \
    -O compression=lz4 \
    -O normalization=formD \
    -O relatime=on \
    -O canmount=off -O mountpoint=/boot -R /mnt \
    bpool ${DISK}3

HTH.

FingerlessGlov3s commented 1 year ago

I recently followed the Bullseye guide for installing bookworm. I didn't have any major problems. I made note of all the commands I ran.

The two things that I did have to do but I don't believe the guide mentions it. After doing the first boot. After doing all the chroot bits. Both the bpool and rpool fail to import because the hostid has changed. So when it drops in to emergency mode on the initramfs. Just do a zpool import -f {poolname} for each pool, then type exit. Then the install works as intended and any reboots after this point boot up as expected.

I have got the bpool talking about doing a pool upgrade, not sure if that happens in Bullseye, but could be a Bookworm thing, I wish I tried the updated bpool creation command @colmbuckley mentioned, but the server I did the install one has hit production now, so can't do a reinstall to test 😅

Enable all features using 'zpool upgrade'. Once this is done,
        the pool may no longer be accessible by software that does not support
        the features. See zpool-features(7) for details.

colmbuckley commented 1 year ago

I wish I tried the updated bpool creation command @colmbuckley mentioned, but the server I did the install one has hit production now, so can't do a reinstall to test 😅

You can still do zpool set compatibility=grub2 bpool on this system, even after installation and boot; that is a very low-risk operation. That should address the messages about upgrading; but it does not itself enable or disable any features, so the warning you quote should not be an issue.

Hope this helps.

FingerlessGlov3s commented 1 year ago

I've found some time to recreate my environment in a VM, and after doing the command you mentioned above, the upgrade warning changes the below.

Inside /usr/share/zfs/compatibility.d/ directory grub2 is listed. Do I need to do something else to mirror the features from the grub2 files exactly?

  pool: bpool
 state: ONLINE
status: One or more features are enabled on the pool despite not being
        requested by the 'compatibility' property.
action: Consider setting 'compatibility' to an appropriate value, or
        adding needed features to the relevant file in
        /etc/zfs/compatibility.d or /usr/share/zfs/compatibility.d.
config:

        NAME                                            STATE     READ WRITE CKSUM
        bpool                                           ONLINE       0     0     0
          mirror-0                                      ONLINE       0     0     0
            scsi-0QEMU_QEMU_HARDDISK_drive-scsi0-part3  ONLINE       0     0     0
            scsi-0QEMU_QEMU_HARDDISK_drive-scsi1-part3  ONLINE       0     0     0

colmbuckley commented 1 year ago

Yes; the instructions added two new features to the bpool pool which are not yet reflected in the grub2 compatibility file - there is a PR for this merged upstream but it hasn't yet been included in a release.

To eliminate this warning (harmlessly), you can do:

# mkdir -p /etc/zfs/compatibility.d
# cp /usr/share/zfs/compatibility.d/grub2 /etc/zfs/compatibility.d
# (echo livelist; echo zpool_checkpoint) >> /etc/zfs/compatibility.d/grub2
# zpool status bpool

FingerlessGlov3s commented 1 year ago

I can confirm doing what you said has fixed the status messages,

When you mean upstream do you mean Debian or openzfs itself?

colmbuckley commented 1 year ago

I mean OpenZFS - this PR. That adds these two features to the system grub2 compatibility file, so there's no need to create a local one (in /etc) as above.

jim-perkins commented 1 year ago

My pull request was merged and there is now Debian 12 Bookworm Root on ZFS documentation. thanks so much. Should this issue be closed and marked as resolved ?

openzfs / openzfs-docs

Debian 12 "bookworm" #442