Reboot is bricked when using GRUB 2 and systemd

[corefx]

It seems that reboot is bricked when using GRUB 2 and systemd.

[unixfox]

Grub 2 wasn't patched because vlany fix only the grub.conf from Grub legacy (1): https://github.com/mempodippy/vlany/commit/47db2c2767f340a7eb23610845f64af980f04269#diff-3fbb47e318cd8802bd325e7da9aaabe8R32 and https://wiki.debian.org/GrubConfiguration

[corefx]

Thanks for explanation @unixfox.

[mempodippy]

I should've explained better... vlany only uses /etc/grub.conf as the default option. For grub2, instead of selecting the default config file, using /boot/grub/grub.cfg will work. (The syntax of the files regardless of version don't change that much, we're only changing boot options.) Edit: I feared this would happen... Figured you actually used /boot/grub/grub.cfg as the config file anyway. So currently the fix might only work for grub legacy boxes, I'll take a little look.

[unixfox]

It seems that the fix doesn't work on Grub2 (or just for me (?)), I specified the right file for Grub2. Here is my grub2 configuration:

#
# DO NOT EDIT THIS FILE
#
# It is automatically generated by grub-mkconfig using templates
# from /etc/grub.d and settings from /etc/default/grub
#

### BEGIN /etc/grub.d/00_header ###
if [ -s $prefix/grubenv ]; then
  set have_grubenv=true
  load_env
fi
if [ "${next_entry}" ] ; then
   set default="${next_entry}"
   set next_entry=
   save_env next_entry
   set boot_once=true
else
   set default="0"
fi

if [ x"${feature_menuentry_id}" = xy ]; then
  menuentry_id_option="--id"
else
  menuentry_id_option=""
fi

export menuentry_id_option

if [ "${prev_saved_entry}" ]; then
  set saved_entry="${prev_saved_entry}"
  save_env saved_entry
  set prev_saved_entry=
  save_env prev_saved_entry
  set boot_once=true
fi

function savedefault {
  if [ -z "${boot_once}" ]; then
    saved_entry="${chosen}"
    save_env saved_entry
  fi
}
function load_video {
  if [ x$feature_all_video_module = xy ]; then
    insmod all_video
  else
    insmod efi_gop
    insmod efi_uga
    insmod ieee1275_fb
    insmod vbe
    insmod vga
    insmod video_bochs
    insmod video_cirrus
  fi
}

if [ x$feature_default_font_path = xy ] ; then
   font=unicode
else
insmod part_gpt
insmod ext2
set root='hd0,gpt1'
if [ x$feature_platform_search_hint = xy ]; then
  search --no-floppy --fs-uuid --set=root --hint-bios=hd0,gpt1 --hint-efi=hd0,gpt1 --hint-baremetal=ahci0,gpt1  11b0b201-d195-42df-89b5-5cc2da17bb67
else
  search --no-floppy --fs-uuid --set=root 11b0b201-d195-42df-89b5-5cc2da17bb67
fi
    font="/usr/share/grub/unicode.pf2"
fi

if loadfont $font ; then
  set gfxmode=auto
  load_video
  insmod gfxterm
fi
terminal_output gfxterm
if [ "${recordfail}" = 1 ] ; then
  set timeout=-1
else
  if [ x$feature_timeout_style = xy ] ; then
    set timeout_style=menu
    set timeout=5
  # Fallback normal timeout code in case the timeout_style feature is
  # unavailable.
  else
    set timeout=5
  fi
fi
### END /etc/grub.d/00_header ###

### BEGIN /etc/grub.d/05_debian_theme ###
set menu_color_normal=cyan/blue
set menu_color_highlight=white/blue
### END /etc/grub.d/05_debian_theme ###

### BEGIN /etc/grub.d/10_linux ###
function gfxmode {
    set gfxpayload="${1}"
}
set linux_gfx_mode=
export linux_gfx_mode
menuentry 'Debian GNU/Linux' --class debian --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-simple-11b0b201-d195-42df-89b5-5cc2da17bb67' {
    load_video
    insmod gzio
    if [ x$grub_platform = xxen ]; then insmod xzio; insmod lzopio; fi
    insmod part_gpt
    insmod ext2
    set root='hd0,gpt1'
    if [ x$feature_platform_search_hint = xy ]; then
      search --no-floppy --fs-uuid --set=root --hint-bios=hd0,gpt1 --hint-efi=hd0,gpt1 --hint-baremetal=ahci0,gpt1  11b0b201-d195-42df-89b5-5cc2da17bb67
    else
      search --no-floppy --fs-uuid --set=root 11b0b201-d195-42df-89b5-5cc2da17bb67
    fi
    echo    'Loading Linux 3.16.0-4-amd64 ...'
    linux   /boot/vmlinuz-3.16.0-4-amd64 root=UUID=11b0b201-d195-42df-89b5-5cc2da17bb67 ro debian-installer=en_US console=tty1 root=LABEL=DOROOT notsc clocksource=kvm-clock net.ifnames=0 quiet
    echo    'Loading initial ramdisk ...'
    initrd  /boot/initrd.img-3.16.0-4-amd64
}
submenu 'Advanced options for Debian GNU/Linux' $menuentry_id_option 'gnulinux-advanced-11b0b201-d195-42df-89b5-5cc2da17bb67' {
    menuentry 'Debian GNU/Linux, with Linux 3.16.0-4-amd64' --class debian --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-3.16.0-4-amd64-advanced-11b0b201-d195-42df-89b5-5cc2da17bb67' {
        load_video
        insmod gzio
        if [ x$grub_platform = xxen ]; then insmod xzio; insmod lzopio; fi
        insmod part_gpt
        insmod ext2
        set root='hd0,gpt1'
        if [ x$feature_platform_search_hint = xy ]; then
          search --no-floppy --fs-uuid --set=root --hint-bios=hd0,gpt1 --hint-efi=hd0,gpt1 --hint-baremetal=ahci0,gpt1  11b0b201-d195-42df-89b5-5cc2da17bb67
        else
          search --no-floppy --fs-uuid --set=root 11b0b201-d195-42df-89b5-5cc2da17bb67
        fi
        echo    'Loading Linux 3.16.0-4-amd64 ...'
        linux   /boot/vmlinuz-3.16.0-4-amd64 root=UUID=11b0b201-d195-42df-89b5-5cc2da17bb67 ro debian-installer=en_US console=tty1 root=LABEL=DOROOT notsc clocksource=kvm-clock net.ifnames=0 quiet
        echo    'Loading initial ramdisk ...'
        initrd  /boot/initrd.img-3.16.0-4-amd64
    }
    menuentry 'Debian GNU/Linux, with Linux 3.16.0-4-amd64 (recovery mode)' --class debian --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-3.16.0-4-amd64-recovery-11b0b201-d195-42df-89b5-5cc2da17bb67' {
        load_video
        insmod gzio
        if [ x$grub_platform = xxen ]; then insmod xzio; insmod lzopio; fi
        insmod part_gpt
        insmod ext2
        set root='hd0,gpt1'
        if [ x$feature_platform_search_hint = xy ]; then
          search --no-floppy --fs-uuid --set=root --hint-bios=hd0,gpt1 --hint-efi=hd0,gpt1 --hint-baremetal=ahci0,gpt1  11b0b201-d195-42df-89b5-5cc2da17bb67
        else
          search --no-floppy --fs-uuid --set=root 11b0b201-d195-42df-89b5-5cc2da17bb67
        fi
        echo    'Loading Linux 3.16.0-4-amd64 ...'
        linux   /boot/vmlinuz-3.16.0-4-amd64 root=UUID=11b0b201-d195-42df-89b5-5cc2da17bb67 ro single debian-installer=en_US console=tty1 root=LABEL=DOROOT notsc clocksource=kvm-clock net.ifnames=0
        echo    'Loading initial ramdisk ...'
        initrd  /boot/initrd.img-3.16.0-4-amd64
    }
}

### END /etc/grub.d/10_linux ###

### BEGIN /etc/grub.d/20_linux_xen ###

### END /etc/grub.d/20_linux_xen ###

### BEGIN /etc/grub.d/30_os-prober ###
### END /etc/grub.d/30_os-prober ###

### BEGIN /etc/grub.d/30_uefi-firmware ###
### END /etc/grub.d/30_uefi-firmware ###

### BEGIN /etc/grub.d/40_custom ###
# This file provides an easy way to add custom menu entries.  Simply type the
# menu entries you want to add after this comment.  Be careful not to change
# the 'exec tail' line above.
### END /etc/grub.d/40_custom ###

### BEGIN /etc/grub.d/41_custom ###
if [ -f  ${config_directory}/custom.cfg ]; then
  source ${config_directory}/custom.cfg
elif [ -z "${config_directory}" -a -f  $prefix/custom.cfg ]; then
  source $prefix/custom.cfg;
fi
### END /etc/grub.d/41_custom ###

Apart from that can you explain what does the fix do? I'm not good at regex and I want to try to find a way to make it work under Grub2.

And watch-out that the file /boot/grub/grub.cfg is regenerated everytime there is an upgrade for grub or for the linux kernel so the modification will not be persistent.

[mempodippy]

From your posted grub.cfg,

root@lol:~# cat lol | grep "\bro\b"
linux/boot/vmlinuz-3.16.0-4-amd64 root=UUID=11b0b201-d195-42df-89b5-5cc2da17bb67 ro debian-installer=en_US console=tty1 root=LABEL=DOROOT notsc clocksource=kvm-clock net.ifnames=0 quiet
linux/boot/vmlinuz-3.16.0-4-amd64 root=UUID=11b0b201-d195-42df-89b5-5cc2da17bb67 ro debian-installer=en_US console=tty1 root=LABEL=DOROOT notsc clocksource=kvm-clock net.ifnames=0 quiet
linux/boot/vmlinuz-3.16.0-4-amd64 root=UUID=11b0b201-d195-42df-89b5-5cc2da17bb67 ro single debian-installer=en_US console=tty1 root=LABEL=DOROOT notsc clocksource=kvm-clock net.ifnames=0
root@lol:~#

using '\b' allows us to find 'ro' on its own, so we're not destroying random stuff. Basically, very early in the boot & init process, vlany can't write to disk thanks to the read only boot options, so we need to alter the config so that vlany can write to disk. Due to the default boot options, when vlany causes something to write to disk at boot, the box seizes up because init doesn't know what to do.

Here's what your posted grub.cfg would look like after being 'patched',

root@lol:~# cat lol | grep "\brw\b"
linux/boot/vmlinuz-3.16.0-4-amd64 root=UUID=11b0b201-d195-42df-89b5-5cc2da17bb67 rw debian-installer=en_US console=tty1 root=LABEL=DOROOT notsc clocksource=kvm-clock net.ifnames=0 quiet
linux/boot/vmlinuz-3.16.0-4-amd64 root=UUID=11b0b201-d195-42df-89b5-5cc2da17bb67 rw debian-installer=en_US console=tty1 root=LABEL=DOROOT notsc clocksource=kvm-clock net.ifnames=0 quiet
linux/boot/vmlinuz-3.16.0-4-amd64 root=UUID=11b0b201-d195-42df-89b5-5cc2da17bb67 rw single debian-installer=en_US console=tty1 root=LABEL=DOROOT notsc clocksource=kvm-clock net.ifnames=0
root@lol:~#

And a patched simple grub.conf file on CentOS 6.6,

[root@killme:~]$ cat /etc/grub.conf
default=0
timeout=10
title vmlinuz-2.6.32-504.el6.x86_64
  root (hd0,0)
  kernel /boot/vmlinuz-2.6.32-504.el6.x86_64 root=LABEL=root rw
  initrd /boot/initramfs-2.6.32-504.el6.x86_64.img
[root@killme:~]$

And thanks for that, I realised. If somebody just decides to update grub.cfg one day using update-grub, vlany's requirements are out the window and the box will be bricked. There's probably some hacky things I can do do intercept this kinda thing.

[unixfox]

Okay thank you, I understand. I think that mounting the kernel to read write doesn't fix the issue on Debian 8 under Grub2, maybe because systemd mounts the /boot differently on Grub2 (?).

[unixfox]

I found that to make it persistent under Debian distributions, just specify /etc/grub.d/10_linux instead of /boot/grub/grub.cfg and then run update-grub. Source: http://askubuntu.com/a/468855/352380

EDIT: The issue about systemd breaking the box is still actually a serious bug because not everybody use grub, there are lot's of bootloaders to cover: https://wiki.archlinux.org/index.php/Category:Boot_loaders I don't know why you removed from the README.

[mempodippy]

Ya, grub.cfg is based on whatever's in grub.d. I mentioned in install.sh, I was just assuming most boxes will come with grub(2) installed. Most bootloaders use similar configuration syntax anyway, so it's just a matter of finding the box's bootloader config(s). Some configs won't even need changed. However, I will amend the README.