AlmaLinux / almalinux-deploy

EL to AlmaLinux migration tool.
GNU General Public License v3.0
542 stars 71 forks source link

README.md should mention disabling Secure Boot as a possibilty to try when reboot fails and it is on. #130

Open kbulgrien opened 2 years ago

kbulgrien commented 2 years ago

After a seemingly problem-free conversion, reboot rendered the system useless until research elsewhere showed that Secure Boot must be disabled. Since CentOS 8 was working fine with the system configured for Secure Boot, the principle of least astonishment should probably drive a conversion document to mention something obvious like this that could make it seem the system was damaged.

At least in my case, entering BIOS and disabling secure boot was enough to get the system functional again, but a lot of time was wasted on this when it could have been addressed in the README.md.

EHRETic commented 2 years ago

Hi,

It's weird, I didn't remove my secure boot for my conversions back at the day. I migrated from/to 8.4 end of last year and I had secure boot activated on all my VMs.

This was also one of the main reasons why I didn't choose Rocky Linux.

My 5 cent for the people that can really help 😉

kbulgrien commented 2 years ago

Weird or not: https://bugs.almalinux.org/view.php?id=3 I see that it is closed, but this does not change the fact that I booted a CentOS 8 that had been updated and described, and that the conversion script ran without mentioning errors. Subsequently, it was rebooted. The HP MP5 system then started a continuous reboot cycle. The cycle was so fast that the logo would barely show. I then proceeded to try a number of things, like download a rescue media and run the rescue operation. In the mean time I searched other resources, and found a page that specifically said to disable Secure Boot.

https://cloudlinuxtech.com/migrate-from-centos-8-to-almalinux-8/

At this point, I turned Secure Boot off. I suppose that it might have been that I ran the rescue operation. I can try turning it back on to see.

There are many such documents. This one references 8.5:

https://techviewleo.com/how-to-convert-centos-to-almalinux-server/

I didn't particularly say a change needed to definitively say to turn off Secure Boot. As far as I am concerned, it would have been helpful to have even the merest suggestion that turning off Secure Boot might be an appropriate action if it happens to be on and a boot problem occurs immediately after migration.

andrewlukoshko commented 2 years ago

Hello. AlmaLinux supports Secure Boot since 8.4 release and almalinux-deploy supports migration of Secure Boot enabled systems too. So could you please help me to understand what went wrong and provide an output of:

uname -a
rpm -qi shim-x64 grub2-efi-x64 kernel

To boot correctly in Secure Boot mode all these packages should be from AlmaLinux after migration. Is it possible that GRUB is trying to boot an older kernel from CentOS?

kbulgrien commented 2 years ago
$ uname -a
Linux hpmp9.home.bulgrien.net 4.18.0-348.20.1.el8_5.x86_64 #1 SMP Thu Mar 10 11:31:47 EST 2022 x86_64 x86_64 x86_64 GNU/Linux

$ rpm -qi shim-x64 grub2-efi-x64 kernel
Name        : shim-x64
Version     : 15.4
Release     : 2.el8_1.alma
Architecture: x86_64
Install Date: Tue 19 Apr 2022 08:49:51 PM CDT
Group       : Unspecified
Size        : 3714330
License     : BSD
Signature   : RSA/SHA256, Thu 20 May 2021 09:44:38 AM CDT, Key ID 51d6647ec21ad6ea
Source RPM  : shim-15.4-2.el8_1.alma.src.rpm
Build Date  : Thu 20 May 2021 09:43:31 AM CDT
Build Host  : vdsl-77.79.198.25.atman.pl
Relocations : (not relocatable)
Packager    : AlmaLinux Packaging Team <packager@almalinux.org>
Vendor      : AlmaLinux
URL         : https://github.com/rhboot/shim/
Summary     : First-stage UEFI bootloader
Description :
Initial UEFI bootloader that handles chaining to a trusted full
bootloader under secure boot environments. This package contains the
version signed by the UEFI signing service.
Name        : grub2-efi-x64
Epoch       : 1
Version     : 2.02
Release     : 106.el8.alma
Architecture: x86_64
Install Date: Tue 19 Apr 2022 08:49:51 PM CDT
Group       : System Environment/Base
Size        : 2291736
License     : GPLv3+
Signature   : RSA/SHA256, Sat 09 Oct 2021 07:32:06 AM CDT, Key ID 51d6647ec21ad6ea
Source RPM  : grub2-2.02-106.el8.alma.src.rpm
Build Date  : Sat 09 Oct 2021 06:26:28 AM CDT
Build Host  : 192-168-251-119.atm.cloudlinux.com
Relocations : (not relocatable)
Packager    : AlmaLinux Packaging Team <packager@almalinux.org>
Vendor      : AlmaLinux
URL         : http://www.gnu.org/software/grub/
Summary     : GRUB for EFI systems.
Description :

The GRand Unified Bootloader (GRUB) is a highly configurable and
customizable bootloader with modular architecture.  It supports a rich
variety of kernel formats, file systems, computer architectures and
hardware devices.

This subpackage provides support for efi-x64 systems.
Name        : kernel
Version     : 4.18.0
Release     : 348.7.1.el8_5
Architecture: x86_64
Install Date: Wed 26 Jan 2022 05:27:02 PM CST
Group       : System Environment/Kernel
Size        : 0
License     : GPLv2 and Redistributable, no modification permitted
Signature   : RSA/SHA256, Wed 22 Dec 2021 08:47:30 AM CST, Key ID 05b555b38483c65d
Source RPM  : kernel-4.18.0-348.7.1.el8_5.src.rpm
Build Date  : Wed 22 Dec 2021 07:45:11 AM CST
Build Host  : kbuilder.bsys.centos.org
Relocations : (not relocatable)
Packager    : CentOS BuildSystem <http://bugs.centos.org>
Vendor      : CentOS
URL         : http://www.kernel.org/
Summary     : The Linux kernel, based on version 4.18.0, heavily modified with backports
Description :
This is the package which provides the Linux kernel for CentOS.
It is based on upstream Linux at version 4.18.0 and maintains kABI
compatibility of a set of approved symbols, however it is heavily modified with
backports and fixes pulled from newer upstream Linux kernel releases. This means
this is not a 4.18.0 kernel anymore: it includes several components which come
from newer upstream linux versions, while maintaining a well tested and stable
core. Some of the components/backports that may be pulled in are: changes like
updates to the core kernel (eg.: scheduler, cgroups, memory management, security
fixes and features), updates to block layer, supported filesystems, major driver
updates for supported hardware in CentOS, enhancements for
enterprise customers, etc.
Name        : kernel
Version     : 4.18.0
Release     : 348.20.1.el8_5
Architecture: x86_64
Install Date: Tue 19 Apr 2022 08:51:02 PM CDT
Group       : System Environment/Kernel
Size        : 0
License     : GPLv2 and Redistributable, no modification permitted
Signature   : RSA/SHA256, Thu 10 Mar 2022 02:24:51 PM CST, Key ID 51d6647ec21ad6ea
Source RPM  : kernel-4.18.0-348.20.1.el8_5.src.rpm
Build Date  : Thu 10 Mar 2022 11:16:28 AM CST
Build Host  : 192-168-251-119.atm.cloudlinux.com
Relocations : (not relocatable)
Packager    : AlmaLinux Packaging Team <packager@almalinux.org>
Vendor      : AlmaLinux
URL         : http://www.kernel.org/
Summary     : The Linux kernel, based on version 4.18.0, heavily modified with backports
Description :
This is the package which provides the Linux kernel for AlmaLinux.
It is based on upstream Linux at version 4.18.0 and maintains kABI
compatibility of a set of approved symbols, however it is heavily modified with
backports and fixes pulled from newer upstream Linux kernel releases. This means
this is not a 4.18.0 kernel anymore: it includes several components which come
from newer upstream linux versions, while maintaining a well tested and stable
core. Some of the components/backports that may be pulled in are: changes like
updates to the core kernel (eg.: scheduler, cgroups, memory management, security
fixes and features), updates to block layer, supported filesystems, major driver
updates for supported hardware in AlmaLinux, enhancements for
enterprise customers, etc.

Sequence of events (shell history):

sudo sed -i -e '/mirrorlist=http:\/\/mirrorlist.centos.org\/?release=$releasever&arch=$basearch&repo=/ s/^#*/#/' -e '/baseurl=http:\/\/mirror.centos.org\/$contentdir\/$releasever\// s/^#*/#/' -e '/^\[baseos\]/a baseurl=https://mirror.rackspace.com/centos-vault/8.5.2111/BaseOS/$basearch/os' /etc/yum.repos.d/CentOS-Linux-BaseOS.repo
sudo sed -i -e '/mirrorlist=http:\/\/mirrorlist.centos.org\/?release=$releasever&arch=$basearch&repo=/ s/^#*/#/' -e '/baseurl=http:\/\/mirror.centos.org\/$contentdir\/$releasever\// s/^#*/#/' -e '/^\[appstream\]/a baseurl=https://mirror.rackspace.com/centos-vault/8.5.2111/AppStream/$basearch/os' /etc/yum.repos.d/CentOS-Linux-AppStream.repo
sudo sed -i -e '/mirrorlist=http:\/\/mirrorlist.centos.org\/?release=$releasever&arch=$basearch&repo=/ s/^#*/#/' -e '/baseurl=http:\/\/mirror.centos.org\/$contentdir\/$releasever\// s/^#*/#/' -e '/^\[cr\]/a baseurl=https://mirror.rackspace.com/centos-vault/8.5.2111/ContinuousRelease/$basearch/os' /etc/yum.repos.d/CentOS-Linux-ContinuousRelease.repo
sudo sed -i -e '/mirrorlist=http:\/\/mirrorlist.centos.org\/?release=$releasever&arch=$basearch&repo=/ s/^#*/#/' -e '/baseurl=http:\/\/mirror.centos.org\/$contentdir\/$releasever\// s/^#*/#/' -e '/^\[devel\]/a baseurl=https://mirror.rackspace.com/centos-vault/8.5.2111/Devel/$basearch/os' /etc/yum.repos.d/CentOS-Linux-Devel.repo
sudo sed -i -e '/mirrorlist=http:\/\/mirrorlist.centos.org\/?release=$releasever&arch=$basearch&repo=/ s/^#*/#/' -e '/baseurl=http:\/\/mirror.centos.org\/$contentdir\/$releasever\// s/^#*/#/' -e '/^\[extras\]/a baseurl=https://mirror.rackspace.com/centos-vault/8.5.2111/extras/$basearch/os' /etc/yum.repos.d/CentOS-Linux-Extras.repo
sudo sed -i -e '/mirrorlist=http:\/\/mirrorlist.centos.org\/?release=$releasever&arch=$basearch&repo=/ s/^#*/#/' -e '/baseurl=http:\/\/mirror.centos.org\/$contentdir\/$releasever\// s/^#*/#/' -e '/^\[fasttrack\]/a baseurl=https://mirror.rackspace.com/centos-vault/8.5.2111/fasttrack/$basearch/os' /etc/yum.repos.d/CentOS-Linux-FastTrack.repo
sudo sed -i -e '/mirrorlist=http:\/\/mirrorlist.centos.org\/?release=$releasever&arch=$basearch&repo=/ s/^#*/#/' -e '/baseurl=http:\/\/mirror.centos.org\/$contentdir\/$releasever\// s/^#*/#/' -e '/^\[ha\]/a baseurl=https://mirror.rackspace.com/centos-vault/8.5.2111/HighAvailability/$basearch/os' /etc/yum.repos.d/CentOS-Linux-HighAvailability.repo
sudo sed -i -e '/mirrorlist=http:\/\/mirrorlist.centos.org\/?release=$releasever&arch=$basearch&repo=/ s/^#*/#/' -e '/baseurl=http:\/\/mirror.centos.org\/$contentdir\/$releasever\// s/^#*/#/' -e '/^\[plus\]/a baseurl=https://mirror.rackspace.com/centos-vault/8.5.2111/centosplus/$basearch/os' /etc/yum.repos.d/CentOS-Linux-Plus.repo
sudo sed -i -e '/mirrorlist=http:\/\/mirrorlist.centos.org\/?release=$releasever&arch=$basearch&repo=/ s/^#*/#/' -e '/baseurl=http:\/\/mirror.centos.org\/$contentdir\/$releasever\// s/^#*/#/' -e '/^\[powertools\]/a baseurl=https://mirror.rackspace.com/centos-vault/8.5.2111/PowerTools/$basearch/os' /etc/yum.repos.d/CentOS-Linux- PowerTools.repo

sudo yum update
curl -O https://raw.githubusercontent.com/AlmaLinux/almalinux-deploy/master/almalinux-deploy.sh
sudo bash almalinux-deploy.sh
sudo reboot

At this point, I use another system to download:

[AlmaLinux-9.0-beta-1-x86_64-boot.iso](http://dal.mirrors.clouvider.net/almalinux/9.0-beta/isos/x86_64/AlmaLinux-9.0-beta-1-x86_64-boot.iso)               18-Apr-2022 19:42           784334848

It doesn't fit on a CD, so I download:

[AlmaLinux-9.0-beta-1-x86_64-minimal.iso](http://dal.mirrors.clouvider.net/almalinux/9.0-beta/isos/x86_64/AlmaLinux-9.0-beta-1-x86_64-minimal.iso)            18-Apr-2022 19:47          1735393280

The image is placed on DVD and loaded in an optical drive via USB. The optical drive had not been installed at the time of the conversion. The system is rebooted, but since while downloading, I found the information about Secure Boot, so before the DVD loads, I enter BIOS and find Secure Boot enabled. I disable it.

Advanced
  Secure Boot Configuration
    Configure Legacy Support and Secure Boot
       Legacy Support Disable and Secure Boot Enable -> Legacy Support Enable and Secure Boot Disable

    Secure Boot Key Management (no changes made)
      [ ] Import Custom Secure Boot keys
      [ ] Clear Secure Boot keys
      [ ] Reset Secure Boot keys to factory defaults
      [x] Enable MS UEFI key

Main
  Save Changes and Exit

Save Changes?
  Yes

I go ahead and let the DVD boot, and for good measure, I let the system repair option run. I imagine this muddied the water somewhat as to which operation fixed the issue.

The system is rebooted without the optical media.

After the system boots, I check kernels present to see how many there are and clear off excess:

uptime
uname -a
rpm -qa | grep kernel
sudo yum erase kernel-4.18.0-240.10.1.el8_3.x86_64
rpm -qa | grep kernel
sudo yum erase kernel-core-4.18.0-240.10.1.el8_3.x86_64 kernel-modules-4.18.0-240.10.1.el8_3.x86_64
sudo yum update

Today I have reversed the setting:

Advanced
  Secure Boot Configuration
    Configure Legacy Support and Secure Boot
       Legacy Support Enable and Secure Boot Disable  ->  Legacy Support Disable and Secure Boot Enable

   Secure Boot Key Management (no changes made)
      [ ] Import Custom Secure Boot keys
      [ ] Clear Secure Boot keys
      [ ] Reset Secure Boot keys to factory defaults
      [x] Enable MS UEFI key

Main
  Save Changes and Exit

Save Changes?
  Yes

The system boots fine. The boot options I have are:

AlmaLinux (4.18.0--348.20.11.el8_5.x86_64) 8.5 (Arctic Sphynx)
CentOS Linux (4.18.0--348.7.1.el8_5.x86_64) 8
CentOS Linux (0-rescue-6c54b9f0571642fe8b2dba27acfb2a90) 8 (Core)
System setup

This may mean the boot issue was not necessarily related to Secure Boot and that the deploy script failed to set up the boot mechanisms properly. It would have been better to try the two different mitigation attempts separately (but as is frequently the case one's goal isn't as much to update a system as to do something else when update problems just get in the way).

-rw-------.  1 root             root              1048549 Apr 19 20:38 dnf.librepo.log.1
-rw-r-----.  1 root             root               445098 Apr 19 21:00 almalinux-deploy.log
-rw-r-----.  1 root             root                75489 Apr 19 21:00 almalinux-deploy.debug.log

almalinux-deploy.log almalinux-deploy.debug.log