Dasharo / dasharo-issues

The Dasharo issue tracker
https://dasharo.com/
24 stars 0 forks source link

External boot file to USB flash drive not found after rebooting #990

Open wessel-novacustom opened 1 month ago

wessel-novacustom commented 1 month ago

Component

Dasharo firmware

Device

NovaCustom V56 14th Gen

Dasharo version

Dasharo (coreboot+UEFI) v0.9.0

Dasharo Tools Suite version

No response

Test case ID

No response

Brief summary

External USB flash drive containing boot directory is not found after rebooting while laptop is connected to powersource and charging, resulting in a reboot failing (/ending up in a shell).

A clear and concise summary of the bug

As an extra security caution I boot all my laptops from an external USB flash drive (/boot/efi is on that drive). I've never experienced any problems on my other laptops with this configuration. On my NovaCustom V56 laptop, there seems to be a bug when I want to reboot the operating system (Pop! OS). When the laptop is disconnected from any charging device (either the power delivery that came with the laptop or a docking station that delivers power to the laptop through the thunderbolt port) the laptop reboots without any problems. However, when the laptop is connected to power (and charging) a reboot fails and ends up in a shell prompt. The reason for that is that the USB flash drive is not found anymore (when entering setup mode, only the SSD is visible, not the flash drive containing the necessary .EFI file).

How reproducible

Bug occurs 100% of the time.

How to reproduce

Step 1. Install OS with boot directory on USB flash drive ( https://www.youtube.com/watch?v=etzJAG_H5F8 ) Step 2. Make sure laptop is connected to a powersource AND charging Step 3. Reboot the operating system (this will fail) Step 4. In the shell, enter "exit" and go to the setup menu Step 5. When for example going to the menu to add a boot option, you will see that only the internal harddisk is shown as an option (the USB flash drive won't be listed) Step 6. Exit the setup menu Step 7. The boot will fail again

(Step 8. As a workaround; remove the USB flash drive when you have returned to the shell and re-insert the USB flash drive. Go to the setup menu again - by typing "exit" in the shell - and now the USB flash drive is visible again and the laptop will boot properly after selecting "continue" in the setup menu.)

Expected behavior

That a reboot from the OS simply reboots the laptop (without having to remove & re-insert the USB flash drive).

Actual behavior

Reboot ends up in a shell because USB Flashdrive is not found by the laptop.

Screenshots

I've made a video of two reboots; one without a charging device that reboots without problems & one with a charging device that fails:

https://github.com/user-attachments/assets/17eaf8f8-08ed-465c-b6a9-56f893229a58

Additional context

Operating system: Pop!_OS 22.04 Kernel: 6.9.3-76060903-generic

Solutions you've tried

The only thing I can do is perform the workaround (Step 8 above) or shut down the laptop completely and restart it.

What would help me a bit, is a setup option for an old-fashioned "bios user password" that has to be entered after turning on the laptop (not the admin password that is used to enter the setup options). This password is also asked upon a reboot, before anything is started. This would make it easier for me to perform the workaround, because I could remove and re-insert the USB flash drive before entering the password and continuing the boot process (most likely resulting in a problem free reboot). However, this is not an option in the current Dasharo coreboot version.

philipandag commented 1 month ago

Running Ubuntu installation on a flashdrive I can't reproduce the issue. The flashdrive is found no matter if the AC is connected or not and the system reboots as expected.

wessel-novacustom commented 1 month ago

Running Ubuntu installation on a flashdrive I can't reproduce the issue. The flashdrive is found no matter if the AC is connected or not and the system reboots as expected.

Ubuntu should be installed on the USB pen drive, running a live USB pen drive is not sufficient to reproduce the issue.

mkopec commented 1 month ago

I believe we intentionally drop removable media from the ESP scanning feature: https://github.com/Dasharo/edk2/blob/26f855c6890743a02961f7b96ffa52e0fc9aa0f4/MdeModulePkg/Library/UefiBootManagerLib/BmBoot.c#L2869-L2891 might be related to this issue

philipandag commented 1 month ago

Ubuntu should be installed on the USB pen drive

That is exactly what I did. Ubuntu was installed on the pendrive, both the boot partition and the root filesystem were installed on a flash drive. The laptop was rebooting to this ubuntu installation as expected.

wessel-novacustom commented 1 month ago

Ubuntu should be installed on the USB pen drive

That is exactly what I did. Ubuntu was installed on the pendrive, both the boot partition and the root filesystem were installed on a flash drive. The laptop was rebooting to this ubuntu installation as expected.

The problem arises when the operating system has been installed on the internal SSD and only the /boot directory is installed on an external USB flashdrive. This is a common security hardening method that makes sure the laptop cannot be booted at all without the USB flashdrive.

Did you do that as well, so only the /boot partition on the USB pen drive?

wessel-novacustom commented 1 month ago

To start my (Pop!) OS I don't use the "boot device" USB (like one would do when booting for example a live-image). My boot device is "Pop!_OS 22.04 LTS" (see attached image) which looks for the \EFI\BOOT\systemd\systemd-bootx64.efi file on the USB flash drive (because that's how I configured Pop! OS during installation).

boot_device

philipandag commented 1 month ago

Did you do that as well, so only the /boot partition on the USB pen drive?

No, I see what you mean. Thanks, I will try to replicate your situation more precisely.

philipandag commented 1 month ago

Despite some troubles with 8GB of ram supposedly not being enough for POP! Os installation I have manged to reproduce your setup as far as I understand.

The root filesystem in installed on an internal nvme drive while the /boot partition resides on an external flash drive. If the flashdrive is not inserted the device won't boot and will enter UEFI shell. The only boot menu entries are POP!_OS 22.04 LTS, Setup, UEFI Shell and the USB flashdrive containing the boot partition. image

Despite that I can't reproduce your issue. The device reboots as expected booting into POP! Os unattendedly every time.

philipandag commented 1 month ago

One of our V56 laptops is now free so I will now repeat the same process on that device.

philipandag commented 1 month ago

On V56TNE the Os also boots properly. The issue does not reproduce on the newest Dasharo version.

philipandag commented 1 month ago

I tried flashing V56TNE with Dasharo v0.9.0 to check if the issue was resolved in one of the release candidates, but I was instructed not to do it and only test on the newest version. After reflashing with v0.9.1-rc2 the issue is appearing.

philipandag commented 1 month ago

This cbmem log shows that what Michał said is true. I wonder why it was not happening before. log_cbmem.txt The DEBUG message from https://github.com/Dasharo/edk2/blob/26f855c6890743a02961f7b96ffa52e0fc9aa0f4/MdeModulePkg/Library/UefiBootManagerLib/BmBoot.c#L2869-L2891 appears in the log.

philipandag commented 3 weeks ago

This issue is reproducing reliably on V560TNE with v0.9.1-rc3 without the need to reproduce the Pop! OS setup. The laptop is not finding a flash drive with Ubuntu live installer after rebooting if the flash drive is stays connected the whole time. I wonder if it is the Ubuntu live image causing that.

wessel-novacustom commented 3 weeks ago

I don't think it's down to Ubunte's live image; you can try rebooting the laptop without power. I suspect that's just fine, which I think is proof that it's not down to the live image (or my Pop OS).

philipandag commented 2 weeks ago

The Ubuntu and Pop! OS live images are not detected only after reboot and only if the AC adapter is connected on v0.9.1-rc3 on V560TND. If the AC adapter is disconnected the drives are recognized properly.

philipandag commented 2 weeks ago

On V560TNE, v0.9.1-rc4 a Sandisk Ultra flash drive with Fedora-Workstation-Live-40 is detected and bootable after reboot with AC either connected or disconnected.

philipandag commented 1 week ago

Now I used two flashdrives on v0.9.1-rc4: one with Dasharo Tools Suite, the other with Ubuntu 22.04 and they are not detected after reboot with AC connected. I don't know why it sometimes happens and sometimes not, but this time I have saved cbmem -1 from reboots with and without AC

cbmem-AC.txt cbmem-no-AC.txt

mkopec commented 5 days ago

Based on the logs it looks like USB detection fails altogether on reboot with AC connected. Not necessarily related to ESP scanning.