No /boot/loader/grub.cfg after reboot

[kanru]

Describe the bug

GRUB boots into grub command line after reboot from update.

To Reproduce

1. Install F40 Silverblue
2. Rebase to F41
3. Run sudo semanage permissive -a bootupd_t to workaround selinux policy issue
4. Run sudo bootupctl update
5. Run rpm-ostree update or overlay any package
6. Reboot

Expected behavior

System should boot as usual.

OS version:

State: idle
BootedDeployment:
● fedora:fedora/41/x86_64/silverblue
                  Version: 41.20240908.n.0 (2024-09-08T08:02:28Z)
               BaseCommit: 4a563b07cb9484d775cf691bdbd57709a5f82ba41ddebb44ab6b9e7de5da5fe8
             GPGSignature: Valid signature by 466CF2D8B60BC3057AA9453ED0622462E99D6AD1
      RemovedBasePackages: noopenh264 0.1.0~openh264_2.4.1-2.fc41
          LayeredPackages: btrbk fish mozilla-openh264 syncthing

Additional context

I was testing bootupd https://fedoraproject.org/wiki/Changes/FedoraSilverblueBootupd#How_To_Test

> sudo bootupctl status
Running as unit: bootupd.service
Component EFI
  Installed: grub2-efi-ia32-1:2.12-4.fc41.x86_64,grub2-efi-x64-1:2.12-4.fc41.x86_64,shim-ia32-15.8-3.x86_64,shim-x64-15.8-3.x86_64
  Update: At latest version
No components are adoptable.
Boot method: EFI

> sudo bootupctl validate
Running as unit: bootupd.service
Validated: EFI

[kanru]

If I read this correctly, ostree skips generating (assumes static config) if it find the string static-configs in /boot/bootupd-state.json

https://github.com/ostreedev/ostree/blob/main/src/libostree/ostree-bootloader-grub2.c#L31-L32

This is my /boot/bootupd-state.json after adopting bootupd

{"installed":{"EFI":{"meta":{"timestamp":"2024-08-08T12:14:11Z","version":"grub2-efi-ia32-1:2.12-4.fc41.x86_64,grub2-efi-x64-1:2.12-4.fc41.x86_64,shim-ia32-15.8-3.x86_64,shim-x64-15.8-3.x86_64"},"filetree":{"children":{"BOOT/BOOTIA32.EFI":{"size":747681,"sha512":"sha512:6d4396f289400516b883733f0fd3621e7ec4d70afd02e988651f37db81298775da69c04b998d87a4760b2fa4b96130c70eb0875fad1cf290c52ea606ae40d12e"},"BOOT/BOOTX64.EFI":{"size":949424,"sha512":"sha512:cc23d8c3cb2dcf749075268b77eb796fb430182cbbc04171ded14d43e32b4a5cdeeb1a08666ee0e288bd37d63f657a9af5e7f2012dd70694d11212d705c60b42"},"BOOT/fbia32.efi":{"size":70360,"sha512":"sha512:c3e94b8d6f07ec9c88b66d48e5b887d29288dd1c9914bf8226af7656e35a023c07a6dbf8a2875994485c348ca358591e02a8eb19574299db3ffb6486697a1568"},"BOOT/fbx64.efi":{"size":87816,"sha512":"sha512:d6107afdaebab6c6f3a9136334b54aeba089a8b40d53f38c74c791dca1274b9d6a02d744a634938516cedfa96a67cb0cff6a59c5537b36faf3c70e3bcee622af"},"fedora/BOOTIA32.CSV":{"size":112,"sha512":"sha512:53a8ec886d24c049ee27f2377de3f02951d537171be1ab8d11459453f094e60ff3667c2b7608581392283643b7864008b7cc4d1cdac6a3ebcab3d0d7d410d9e1"},"fedora/BOOTX64.CSV":{"size":110,"sha512":"sha512:0c29b8ae73171ef683ba690069c1bae711e130a084a81169af33a83dfbae4e07d909c2482dbe89a96ab26e171f17c53f1de8cb13d558bc1535412ff8accf253f"},"fedora/grubia32.efi":{"size":3022144,"sha512":"sha512:a81b24b44fdd646d036e963e54dcc76b9465798209f02c89b4cf528966ecd72dbcac78f733b3fd177f0f48091b7605a48eaff0663768ea72b962e123065a2b88"},"fedora/grubx64.efi":{"size":4066624,"sha512":"sha512:2981b7fe1c22969605104bab9ae6f45696c81412b0c08afb78827bc96dc8b53279661b4b9926679e864a7427bbf183169538aa78d3c29eafcc7468ffd70aa721"},"fedora/mmia32.efi":{"size":673992,"sha512":"sha512:cd2cd82de67d2176d212fbe99caa6ef621f971b0f49f1fe7430dd1d4f1bf822fe8f87fcbf5a23512dbb8a9c90c75d5e2ff0e8f910954c62749ef35237e48720a"},"fedora/mmx64.efi":{"size":848080,"sha512":"sha512:d62e003cae3321b3b0aea35c27868176a867b06a1a1eb3266efe84f72f01db6ce71c36073afc192500385743b5eb3d1f35519729b51d0326cebb1e3a04b24281"},"fedora/shim.efi":{"size":949424,"sha512":"sha512:cc23d8c3cb2dcf749075268b77eb796fb430182cbbc04171ded14d43e32b4a5cdeeb1a08666ee0e288bd37d63f657a9af5e7f2012dd70694d11212d705c60b42"},"fedora/shimia32.efi":{"size":747681,"sha512":"sha512:6d4396f289400516b883733f0fd3621e7ec4d70afd02e988651f37db81298775da69c04b998d87a4760b2fa4b96130c70eb0875fad1cf290c52ea606ae40d12e"},"fedora/shimx64.efi":{"size":949424,"sha512":"sha512:cc23d8c3cb2dcf749075268b77eb796fb430182cbbc04171ded14d43e32b4a5cdeeb1a08666ee0e288bd37d63f657a9af5e7f2012dd70694d11212d705c60b42"}}},"adopted-from":{"timestamp":"2024-06-25T13:02:47.923993936Z","version":"unknown"}}},"pending":null,"static-configs":null}

Note the "static-configs":null part. I suspect this is why my grub.cfg was not regenerated. It should be reproducible in a VM, I'll try that next.

[travier]

OK, this looks like an ostree bug. Did you have BLS config disabled in your GRUB config?

You can check that in /etc/default/grub:

$ grep GRUB_ENABLE_BLSCFG /etc/default/grub

[travier]

I've filed: https://github.com/ostreedev/ostree/issues/3295

[kanru]

GRUB_ENABLE_BLSCFG=true

But I think grub still needs the config the load BLS. My grub.cfg was completely missing so that didn't help 😆

It looks like F41 already have the static configs enabled, so this might only affect systems upgraded from F40.

[kanru]

I reproduced it in a VM and updated the description.

[travier]

"Fix/workaround" in https://github.com/ostreedev/ostree/pull/3300

[travier]

Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2312453

[jvnknvlgl]

I'm also encountering this, reading and checking documentation is apparently hard, sorry about that. Is there a way I can recover from this, preferably without a reinstallation?

[travier]

You can copy a default GRUB config in /boot where the config is expected:

#
# DO NOT EDIT THIS FILE
#
# It is automatically generated by grub2-mkconfig using templates
# from /etc/grub.d and settings from /etc/default/grub
#

### BEGIN /etc/grub.d/00_header ###
set pager=1

if [ -f ${config_directory}/grubenv ]; then
  load_env -f ${config_directory}/grubenv
elif [ -s $prefix/grubenv ]; then
  load_env
fi
if [ "${next_entry}" ] ; then
   set default="${next_entry}"
   set next_entry=
   save_env next_entry
   set boot_once=true
else
   set default="${saved_entry}"
fi

if [ x"${feature_menuentry_id}" = xy ]; then
  menuentry_id_option="--id"
else
  menuentry_id_option=""
fi

export menuentry_id_option

if [ "${prev_saved_entry}" ]; then
  set saved_entry="${prev_saved_entry}"
  save_env saved_entry
  set prev_saved_entry=
  save_env prev_saved_entry
  set boot_once=true
fi

function savedefault {
  if [ -z "${boot_once}" ]; then
    saved_entry="${chosen}"
    save_env saved_entry
  fi
}

function load_video {
  if [ x$feature_all_video_module = xy ]; then
    insmod all_video
  else
    insmod efi_gop
    insmod efi_uga
    insmod ieee1275_fb
    insmod vbe
    insmod vga
    insmod video_bochs
    insmod video_cirrus
  fi
}

terminal_output console
if [ x$feature_timeout_style = xy ] ; then
  set timeout_style=menu
  set timeout=5
# Fallback normal timeout code in case the timeout_style feature is
# unavailable.
else
  set timeout=5
fi
### END /etc/grub.d/00_header ###

### BEGIN /etc/grub.d/01_users ###
if [ -f ${prefix}/user.cfg ]; then
  source ${prefix}/user.cfg
  if [ -n "${GRUB2_PASSWORD}" ]; then
    set superusers="root"
    export superusers
    password_pbkdf2 root ${GRUB2_PASSWORD}
  fi
fi
### END /etc/grub.d/01_users ###

### BEGIN /etc/grub.d/08_fallback_counting ###
insmod increment
# Check if boot_counter exists and boot_success=0 to activate this behaviour.
if [ -n "${boot_counter}" -a "${boot_success}" = "0" ]; then
  # if countdown has ended, choose to boot rollback deployment,
  # i.e. default=1 on OSTree-based systems.
  if  [ "${boot_counter}" = "0" -o "${boot_counter}" = "-1" ]; then
    set default=1
    set boot_counter=-1
  # otherwise decrement boot_counter
  else
    decrement boot_counter
  fi
  save_env boot_counter
fi
### END /etc/grub.d/08_fallback_counting ###

### BEGIN /etc/grub.d/10_linux ###
insmod part_gpt
insmod ext2
search --no-floppy --fs-uuid --set=root de867ef3-5bb9-4668-aba6-fd64cecb854e
insmod part_gpt
insmod fat
search --no-floppy --fs-uuid --set=boot 68D7-093A

# This section was generated by a script. Do not modify the generated file - all changes
# will be lost the next time file is regenerated. Instead edit the BootLoaderSpec files.
#
# The blscfg command parses the BootLoaderSpec files stored in /boot/loader/entries and
# populates the boot menu. Please refer to the Boot Loader Specification documentation
# for the files format: https://systemd.io/BOOT_LOADER_SPECIFICATION/.

# The kernelopts variable should be defined in the grubenv file. But to ensure that menu
# entries populated from BootLoaderSpec files that use this variable work correctly even
# without a grubenv file, define a fallback kernelopts variable if this has not been set.
#
# The kernelopts variable in the grubenv file can be modified using the grubby tool or by
# executing the grub2-mkconfig tool. For the latter, the values of the GRUB_CMDLINE_LINUX
# and GRUB_CMDLINE_LINUX_DEFAULT options from /etc/default/grub file are used to set both
# the kernelopts variable in the grubenv file and the fallback kernelopts variable.
if [ -z "${kernelopts}" ]; then
  set kernelopts="root=UUID=3151c4f0-fd30-4a8d-a809-ddafc595e9d7 ro rd.luks.uuid=luks-f8c70476-0125-4ce4-ae0b-8a7f0fc24fd6 rhgb quiet "
fi

insmod blscfg
blscfg
### END /etc/grub.d/10_linux ###

### BEGIN /etc/grub.d/10_reset_boot_success ###
# Hiding the menu is ok if last boot was ok or if this is a first boot attempt to boot the entry
if [ "${boot_success}" = "1" -o "${boot_indeterminate}" = "1" ]; then
  set menu_hide_ok=1
else
  set menu_hide_ok=0 
fi
# Reset boot_indeterminate after a successful boot
if [ "${boot_success}" = "1" ] ; then
  set boot_indeterminate=0
# Avoid boot_indeterminate causing the menu to be hidden more than once
elif [ "${boot_indeterminate}" = "1" ]; then
  set boot_indeterminate=2
fi
# Reset boot_success for current boot 
set boot_success=0
save_env boot_success boot_indeterminate
### END /etc/grub.d/10_reset_boot_success ###

### BEGIN /etc/grub.d/12_menu_auto_hide ###
if [ x$feature_timeout_style = xy ] ; then
  if [ "${menu_show_once}" ]; then
    unset menu_show_once
    save_env menu_show_once
    set timeout_style=menu
    set timeout=60
  elif [ "${menu_auto_hide}" -a "${menu_hide_ok}" = "1" ]; then
    set orig_timeout_style=${timeout_style}
    set orig_timeout=${timeout}
    if [ "${fastboot}" = "1" ]; then
      # timeout_style=menu + timeout=0 avoids the countdown code keypress check
      set timeout_style=menu
      set timeout=0
    else
      set timeout_style=hidden
      set timeout=1
    fi
  fi
fi
### END /etc/grub.d/12_menu_auto_hide ###

### BEGIN /etc/grub.d/14_menu_show_once ###
if [ x$feature_timeout_style = xy ]; then
  if [ "${menu_show_once_timeout}" ]; then
    set timeout_style=menu
    set timeout="${menu_show_once_timeout}"
    unset menu_show_once_timeout
    save_env menu_show_once_timeout
  fi
fi
### END /etc/grub.d/14_menu_show_once ###

### BEGIN /etc/grub.d/15_ostree ###
### END /etc/grub.d/15_ostree ###

### BEGIN /etc/grub.d/20_linux_xen ###
### END /etc/grub.d/20_linux_xen ###

### BEGIN /etc/grub.d/20_ppc_terminfo ###
### END /etc/grub.d/20_ppc_terminfo ###

### BEGIN /etc/grub.d/30_os-prober ###
### END /etc/grub.d/30_os-prober ###

### BEGIN /etc/grub.d/30_uefi-firmware ###
if [ "$grub_platform" = "efi" ]; then
        menuentry 'UEFI Firmware Settings' $menuentry_id_option 'uefi-firmware' {
                fwsetup
        }
fi
### END /etc/grub.d/30_uefi-firmware ###

### BEGIN /etc/grub.d/35_fwupd ###
### END /etc/grub.d/35_fwupd ###

### BEGIN /etc/grub.d/40_custom ###
# This file provides an easy way to add custom menu entries.  Simply type the
# menu entries you want to add after this comment.  Be careful not to change
# the 'exec tail' line above.
### END /etc/grub.d/40_custom ###

### BEGIN /etc/grub.d/41_custom ###
if [ -f  ${config_directory}/custom.cfg ]; then
  source ${config_directory}/custom.cfg
elif [ -z "${config_directory}" -a -f  $prefix/custom.cfg ]; then
  source $prefix/custom.cfg
fi
### END /etc/grub.d/41_custom ###

[jvnknvlgl]

Managed to fix it, thank you very much! Am I correct in assuming that .8 contains the fix?

[travier]

I don't understand what you mean by .8 but Silverblue 41.20240922.n.0 at least has the fix for this issue.

[jvnknvlgl]

I meant ostree, but that confirms it, thanks.

[nikodunk]

Anecdotal, but just updated my ublue bluefin 40 machine to silverblue 41 beta according to the Fedora Magazine article, and worked flawlessly. The previous time I tried this a few weeks ago, I ran into the 'No /boot/loader/grub.cfg' bug, so I'm adding a datapoint that it's resolved and now smooth. Thanks for all the hard work!!