Btrfs guide

[TheChymera]

@Doeme, could you write down the instructions for enabling btrfs management of the images? I could not find them anywhere in the docs.

[Doeme]

What do you mean? Building an image with BTRFS? We have to change a configuration variable in roots/<ID>/conf/ or in the global configuration.

Generally, the documentation is imho an unsolved problem. It would probably be best to add some in-line documentation to the scripts//.sh files, where we can specify what configuration options we use, similar to the eclass documentation.

[TheChymera]

No, I meant setting up btrfs on an OpenStack system in order to more compactly manage images for multiple specializations. I know we did it together once and it was surprisingly easy.

[Doeme]

Ah, that basically was just growing the filesystem (it works without doing that, but is rather inefficient).

1. use fdisk or another partition tool to grow the partition (should work with deleting partition 1 and adding a new one that spans the whole disk)
2. https://btrfs.wiki.kernel.org/index.php/UseCases#Growing

[TheChymera]

Can I do this while the partition is active? this is the system's root partition after all:

bs1 /usr/share/gebuilder/scripts # df -h
Filesystem      Size  Used Avail Use% Mounted on
devtmpfs         10M     0   10M   0% /dev
tmpfs            16G     0   16G   0% /dev/shm
tmpfs            16G  364K   16G   1% /run
/dev/vda1        99G   19G   76G  21% /
cgroup_root      10M     0   10M   0% /sys/fs/cgroup

[Doeme]

Yes, thats the fancy thing about BTRFS, you can resize it online

[TheChymera]

And that also reformats it? Based on /etc/fstab it seems to me that the partition is ext4, and btrfs-convert /dev/vda1 failed on account of the device being mounted.

[Doeme]

Well, was the image built with the BTRFS-configuration set?

[TheChymera]

I don't know, I just built it with gebuild stemgentoo initialize. The string btrfs seems to only appear in this package as a kernel option, and that is always set, but I am guessing that only controls whether the kernel could boot btrfs if the system was btrfs. Is something else happening?

[TheChymera]

When I edit /usr/share/gebuilder/roots/stemgentoo/config/openstack.conf to include:

OPENSTACK_FILESYSTEM="btrfs"

the openstack_image subcommand, gives me:

[...]
Formatting the disk with btrfs
ERROR: unrecognized filesystem feature '^64bit'
[...]

This can be fixed by setting:

OPENSTACK_FILESYSTEM_OPTS=""

[TheChymera]

Also, I can't seem to boot images created with:

OPENSTACK_IMAGE_NAME="image_$(date "+%Y%m%d")"
OPENSTACK_ROOT_PASSWORD="t00r"
OPENSTACK_FILESYSTEM="btrfs"

#Syslinux can't handle 64bit, so we disable it.
#This has the effect, that our root partition can't grow larger than 2TB
OPENSTACK_FILESYSTEM_OPTS=""

[TheChymera]

@Doeme

I have also tried setting:

scripts/openstack_image/stemgentoo/40-generate_bootchain.sh
18:      LINUX /boot/vmlinuz root=UUID=$OPENSTACK_IMG_UUID rootfstype=btrfs console=ttyS0,115200n8
24:UUID=$UUID              /               btrfs            noatime         0 1

since they were previously set to ext4, but this did not help either.

[Doeme]

Either there is no /boot/{sys,ext}linux/{sys,ext}linux.cfg present, or syslinux does not find the partition it has to boot from.

[TheChymera]

So it seems the file exists on the host system, but not on the stemgentoo system. Is this the issue?

bs1 /usr/share/gebuilder/roots/stemgentoo # stat root/boot/{sys,ext}linux/{sys,ext}linux.cfg
stat: cannot stat 'root/boot/syslinux/syslinux.cfg': No such file or directory
stat: cannot stat 'root/boot/syslinux/extlinux.cfg': No such file or directory
stat: cannot stat 'root/boot/extlinux/syslinux.cfg': No such file or directory
stat: cannot stat 'root/boot/extlinux/extlinux.cfg': No such file or directory
bs1 /usr/share/gebuilder/roots/stemgentoo # stat /boot/{sys,ext}linux/{sys,ext}linux.cfg
  File: /boot/syslinux/syslinux.cfg
  Size: 169             Blocks: 8          IO Block: 4096   regular file
Device: fe01h/65025d    Inode: 225091      Links: 1
Access: (0644/-rw-r--r--)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2018-01-31 03:21:39.156000000 +0000
Modify: 2018-01-31 03:21:39.156000000 +0000
Change: 2018-01-31 03:21:39.156000000 +0000
 Birth: -
stat: cannot stat '/boot/syslinux/extlinux.cfg': No such file or directory
stat: cannot stat '/boot/extlinux/syslinux.cfg': No such file or directory
stat: cannot stat '/boot/extlinux/extlinux.cfg': No such file or directory

[Doeme]

This file only ever exists (if at all) in the image file, so you probably have to mount it with mount -t btrf -o loop /path/to/roots/<ID>/<cantrememberwhatgoesintohere>/image_<date> /path/to/mountpoint

[TheChymera]

@Doeme you seem to only have enabled the main BTRFS kernel option, do you think enabling any of the sub-options might help?

utils/openstack_kernel.config
2243:CONFIG_BTRFS_FS=y
2244:# CONFIG_BTRFS_FS_POSIX_ACL is not set
2245:# CONFIG_BTRFS_FS_CHECK_INTEGRITY is not set
2246:# CONFIG_BTRFS_FS_RUN_SANITY_TESTS is not set
2247:# CONFIG_BTRFS_DEBUG is not set
2248:# CONFIG_BTRFS_ASSERT is not set

[TheChymera]

Also, the image doesn't seem to mount:

bs1 ~ # mount -t btrfs -o loop /usr/share/gebuilder/roots/stemgentoo/openstack_images/image_20180219 /mnt/debug
mount: /mnt/debug: wrong fs type, bad option, bad superblock on /dev/loop2, missing codepage or helper program, or other error.

And here's the log:

https://bpaste.net/show/d9e4694d4845

And there's some BTRFS stuff in dmesc, but it looks like it's from the build process rather than the mount attempt:

[   73.292293] random: crng init done
[ 1048.343651] EXT4-fs (dm-1): mounted filesystem with ordered data mode. Opts: (null)
[ 4143.677206] EXT4-fs (loop2p1): mounted filesystem with ordered data mode. Opts: (null)
[ 4755.661484] EXT4-fs (loop2p1): mounted filesystem with ordered data mode. Opts: (null)
[16322.677894] EXT4-fs (loop2p1): mounted filesystem with ordered data mode. Opts: (null)
[17792.672314] blk_update_request: I/O error, dev loop2, sector 10485632
[17792.672319] blk_update_request: I/O error, dev loop2, sector 10485632
[17792.672321] Buffer I/O error on dev loop2p1, logical block 1310448, async page read
[18121.668189] EXT4-fs (loop2): VFS: Can't find ext4 filesystem
[19780.720997] BTRFS: device label NeuroGentoo devid 1 transid 5 /dev/loop2p1
[19780.775805] BTRFS info (device loop2p1): disk space caching is enabled
[19780.775806] BTRFS info (device loop2p1): has skinny extents
[19780.775807] BTRFS info (device loop2p1): flagging fs with big metadata feature
[19780.778540] BTRFS info (device loop2p1): creating UUID tree

[TheChymera]

Ok, so I finally got it to mount, and syslinux.cfg is present:

bs1 ~ # losetup -D
bs1 ~ # losetup -fP --show /usr/share/gebuilder/roots/stemgentoo/openstack_images/image_20180219
/dev/loop2
bs1 ~ # mount /dev/loop2p0 /mnt/debug
mount: /mnt/debug: special device /dev/loop2p0 does not exist.
bs1 ~ # mount /dev/loop2p1 /mnt/debug
bs1 ~ # ls /mnt/debug/boot/
.keep                        config-4.9.76-gentoo-r1      initramfs-4.9.76-gentoo      vmlinuz
System.map-4.9.76-gentoo-r1  initramfs                    syslinux/                    vmlinuz-4.9.76-gentoo-r1
bs1 ~ # ls /mnt/debug/boot/syslinux/
ldlinux.c32  libcom32.c32  libutil.c32  memdisk  menu.c32  syslinux.cfg
bs1 ~ # cat /mnt/debug/boot/syslinux/syslinux.cfg
DEFAULT gentoo
LABEL gentoo
      LINUX /boot/vmlinuz root=UUID=dbead97f-412e-4017-a796-272506130034 rootfstype=btrfs console=ttyS0,115200n8
      INITRD /boot/initramfs

which isn't much different from what's running on the host system:

bs1 ~ # cat /boot/syslinux/syslinux.cfg
DEFAULT gentoo
LABEL gentoo
      LINUX /boot/vmlinuz root=UUID=1c7b3b02-6831-4e1f-9dc1-7d9f72ad2356 rootfstype=ext4 console=ttyS0,115200n8
      INITRD /boot/initramfs

[TheChymera]

Strange thing is, this fails even if I revert to ext4 now, and to be honest, I haven't re-created the system from scratch ever since we migrated from the NeuroGentoo repo:

bs1 # losetup -fP --show /usr/share/gebuilder/roots/stemgentoo/openstack_images/image_20180220
/dev/loop2
bs1 # mount /dev/loop2p1 /mnt/debug
bs1 ~ # cat /mnt/debug/etc/fstab                                                                                                                                                                                                  
# /etc/fstab: static file system information.
#
# noatime turns off atimes for increased performance (atimes normally aren't
# needed); notail increases performance of ReiserFS (at the expense of storage
# efficiency).  It's safe to drop the noatime options if you want and to
# switch between notail / tail freely.
#
# The root filesystem should have a pass number of either 0 or 1.
# All other filesystems should have a pass number of 0 or greater than 1.
#
# See the manpage fstab(5) for more information.
#

# <fs>                  <mountpoint>    <type>          <opts>          <dump/pass>

# NOTE: If your BOOT partition is ReiserFS, add the notail option to opts.
#
# NOTE: Even though we list ext4 as the type here, it will work with ext2/ext3
#       filesystems.  This just tells the kernel to use the ext4 driver.
#
# NOTE: You can use full paths to devices like /dev/sda3, but it is often
#       more reliable to use filesystem labels or UUIDs. See your filesystem
#       documentation for details on setting a label. To obtain the UUID, use
#       the blkid(8) command.

#LABEL=boot             /boot           ext4            noauto,noatime  1 2
#UUID=58e72203-57d1-4497-81ad-97655bd56494              /               ext4            noatime         0 1
#LABEL=swap             none            swap            sw              0 0
#/dev/cdrom             /mnt/cdrom      auto            noauto,ro       0 0
UUID=              /               ext4            noatime         0 1
bs1 ~ # cat /mnt/debug/boot/syslinux/syslinux.cfg
DEFAULT gentoo
LABEL gentoo
      LINUX /boot/vmlinuz root=UUID=5983e763-060d-45a1-bb45-b22437ba8874 rootfstype=ext4 console=ttyS0,115200n8
      INITRD /boot/initramfs

So it's entirely possible that something broke in the mean time. Any idea what that could be?

[TheChymera]

Just for reference, this was all happening before and is still happening after c10997816c6788ce321a3995fa85833013939759 . And to be frank, I have no idea how btrfs could have worked before this (unless the relevant config files are outright ignored for some reason).

[TheChymera]

@Doeme I looked through the entire diff, and spotted nothing suspect:

http://bpaste.net/show/cd2af47506c2

Do you see anything that might be causing this issue?

[Doeme]

the rootfstype option is, I think, not critical since the kernel iterates through all available fstypes. It is, afaik, only a hint on what to try first. The fstab should also be (somewhat) irrelevant, since the kernel has / mounted anyways, and the fstab-entry is only used for fsck or other maintenance operations.

I do not know what changed, but the process of installing the bootloader is here:

https://github.com/IBT-FMI/gebuilder/blob/master/gebuilder/scripts/openstack_image/default/40-generate_bootchain.sh

maybe look at the logfile /roots//logs/openstack_image/40-generate_bootchain.sh.log (or similar)

[TheChymera]

While a major issue (ideally I would want to reference a working project in the pioneer fellowship submission - deadline next week), this seems unrelated to btrfs. Continued in the linked discussion.

[TheChymera]

@Doeme please have a look at whether anything is still missing from the relevant README section.