search

oamg / leapp-repository

Leapp repositories containing actors for the Leapp framework (https://github.com/oamg/leapp). Currently provides leapp repositories for in-place upgrades of RHEL systems.
Apache License 2.0
48 stars 144 forks source link

RHEL8 with EFI after upgrade to 9.x doesn't boot, because of missed GRUB config file #1261

Open yuravk opened 1 month ago

yuravk commented 1 month ago

Actual behavior System doesn't boot because of missed /boot/grub2/grub.cfg GRUB configuration file, after upgrading of recent Red Hat Enterprise Linux release 8.10 to 9.4, and if Extensible Firmware Interface (EFI) is used.

To Reproduce

  1. Get system with most recent RHEL8 and EFI. Register it with RHSM, upgrade, disable zone drifting in the firewall.
  2. Setup Leapp
  3. Pre-upgrade check with Leapp, review results
  4. Upgrade with leapp upgrade --debug
  5. Reboot the system
  6. Get into the system's main console, wait for Leapp to do the upgrade
  7. After the upgrade, Leapp will reboot the system, and grub will enter its CLI because of missed configuration file (see screenshot attached)

Expected behaviour

Assuming a kind of "emergency shell", and chroot into the broken system

[root@localhost-live /]# ls -la /boot/grub2/grub.cfg ls: cannot access '/boot/grub2/grub.cfg': No such file or directory

Upgrade worked till _Red Hat Enterprise Linux release 9.4 (Plow)_ and **GRUB** version **2.06-80.el9_4**.

Leapp should correctly handle such configuration.

**System information (please complete the following information):**

- OS and version:

cat /etc/redhat-release

Red Hat Enterprise Linux release 8.10 (Ootpa)

- system information

uname -a

Linux rhel8.tst 4.18.0-553.8.1.el8_10.x86_64 #1 SMP Fri Jun 14 03:19:37 EDT 2024 x86_64 x86_64 x86_64 GNU/Linux

- EFI configuration

efibootmgr

BootCurrent: 0004 BootOrder: 0004,0000,0001,0002,0003 Boot0000 EFI Virtual disk (0.0) Boot0001 EFI VMware Virtual SATA CDROM Drive (0.0) Boot0002 EFI Network Boot0003 EFI Internal Shell (Unsupported option) Boot0004* Red Hat Enterprise Linux

- installed leapp packages

rpm -qa "leapp"

python3-leapp-0.17.0-1.el8.noarch leapp-upgrade-el8toel9-0.20.0-2.el8.noarch leapp-deps-0.17.0-1.el8.noarch leapp-0.17.0-1.el8.noarch leapp-upgrade-el8toel9-deps-0.20.0-2.el8.noarch

- RHSM information

subscription-manager list

+-------------------------------------------+ Installed Product Status +-------------------------------------------+ Product Name: Red Hat Enterprise Linux for x86_64 Product ID: 479 Version: 8.10 Arch: x86_64 Status: Subscribed Status Details: Starts: xx/xx/xxxx Ends: xx/xx/xxxx


- GRUB configuration files on the system:

ls -la /etc/grub2-efi.cfg

lrwxrwxrwx. 1 root root 31 Feb 21 15:12 /etc/grub2-efi.cfg -> ../boot/efi/EFI/redhat/grub.cfg

ls -la /boot/efi/EFI/redhat/grub.cfg

-rwx------. 1 root root 6645 May 29 11:01 /boot/efi/EFI/redhat/grub.cfg



**Attachments**

<img width="818" alt="Screenshot 2024-07-03 at 16 51 32" src="https://github.com/oamg/leapp-repository/assets/1556475/f406e154-e046-40d7-8547-50f12ecf6bea">

[leapp-logs.tar.gz](https://github.com/user-attachments/files/16086412/leapp-logs.tar.gz)
pirat89 commented 1 month ago

hI @yuravk , this is actually bug in grub2 on rhel9.4. We are waiting for the fix.

morete commented 3 weeks ago

Hi @pirat89 , is there a jira at redhat that could be tracked for progress/status? I just lifted our upgrade path from 8.8 > 9.2 to 8.10 > 9.4 and I'm hitting this issue. I tried the proposed fixing actor in https://github.com/oamg/leapp-repository/pull/1229 but it did not solve it... Thanks!

pirat89 commented 3 weeks ago

@morete Hi. I just returned from vacation. yes, it is tracked here: https://issues.redhat.com/browse/RHEL-40362 however I see it has restricted access :shrug: . Here is related KCS article: https://access.redhat.com/solutions/7075830

We will leave here a new update when we have more info. Currently we have internally new grub2 build available for testing which should fix this behaviour. So far, fixed grub2 package is expected to be released earlier than leapp-repository, that's why we have not merged any changes in this repository yet.

Also the #1229 PR has been developed for a little bit different issue. Unfortunately, the fix in grub2 introduced a different problems for UEFI. That's why it's affecting now larger set of machines.

pirat89 commented 2 weeks ago

We can confirm the problem is going to be fixed with the next grub2 9.4.z-stream release. Upgrades are passing with the new build as expected.

morete commented 2 weeks ago

@pirat89 thanks for the update. Few weeks back I managed to see the EFI grub.cfg file after the failed upgrade is trying to source the old BIOS grub.cfg file location (which we no longer use, so does not exist) so I did workaround it by changing the automation code to create the grub config on both locations BIOS and EFI config file paths prior to running the leapp upgrade command and once the upgrade is finished then the automation reverts the workaround by deleting the old BIOS grub.cfg and updating the EFI grub.cfg... that made all work as expected so with that "temporal" workaround the upgrade works fine...

pirat89 commented 2 weeks ago

@morete thanks for the info, yes that should be possible workaround for now.

yuravk commented 2 weeks ago

I can confirm the issue was fixed for me with grub2 2.06-82.el9 packages.