search

Dasharo / dasharo-issues

The Dasharo issue tracker
https://dasharo.com/
25 stars 0 forks source link

Unable to boot DTS v2.0.0-rc7 in QEMU #1064

Open pietrushnic opened 1 month ago

pietrushnic commented 1 month ago

Component

Dasharo Tools Suite

Device

QEMU Q35 Emulator

Dasharo version

I'm using standard OVMF

Dasharo Tools Suite version

v2.0.0-rc7 (this field seem to be redunant when components is DTS)

Test case ID

No response

Brief summary

System hangs no matter which *.efi file I try to use.

How reproducible

Always

How to reproduce

  1. Download: https://github.com/Dasharo/meta-dts/releases/tag/v2.0.0-rc7
  2. ungzip
  3. Run command:
qemu-system-x86_64 \
>   -drive if=pflash,format=raw,readonly=on,file=Build/OvmfX64/DEBUG_GCC5/FV/OVMF_CODE.fd \
>   -drive if=pflash,format=raw,file=Build/OvmfX64/DEBUG_GCC5/FV/OVMF_VARS.fd \
>   -net none \
>   -nographic \
>   -chardev file,path=debug.log,id=ovmf-debug -device isa-debugcon,iobase=0x402,chardev=ovmf-debug \
>   -device qemu-xhci,id=xhci -M q35 -m 2G \
>   -drive file=../dts-base-image-v2.0.0-rc7.wic,format=raw

YMMV in case of path fro OVMF.

Expected behavior

System boots and DTS menu is present.

Actual behavior

BdsDxe: loading Boot0001 "EFI Internal Shell" from Fv(7CB8BDC9-F8EB-4F34-AAEA-3EE4AF6516A1)/FvFile(7C04A583-9E3E-4F1C-AD65-E05268D0B4D1)
BdsDxe: starting Boot0001 "EFI Internal Shell" from Fv(7CB8BDC9-F8EB-4F34-AAEA-3EE4AF6516A1)/FvFile(7C04A583-9E3E-4F1C-AD65-E05268D0B4D1)
UEFI Interactive Shell v2.2
EDK II
UEFI v2.70 (EDK II, 0x00010000)
Mapping table
      FS0: Alias(s):HD0a65535a1:;BLK1:
          PciRoot(0x0)/Pci(0x1F,0x2)/Sata(0x0,0xFFFF,0x0)/HD(1,MBR,0x076C4A2A,0x800,0xF114)
     BLK0: Alias(s):
          PciRoot(0x0)/Pci(0x1F,0x2)/Sata(0x0,0xFFFF,0x0)
     BLK3: Alias(s):
          PciRoot(0x0)/Pci(0x1F,0x2)/Sata(0x2,0xFFFF,0x0)
     BLK2: Alias(s):
          PciRoot(0x0)/Pci(0x1F,0x2)/Sata(0x0,0xFFFF,0x0)/HD(2,MBR,0x076C4A2A,0x10000,0x200000)
Press ESC in 4 seconds to skip startup.nsh or any other key to continue.
Shell> fs0:
FS0:\> \efi\DTS\grubx64.efi
error: no such device: ((hd0,msdos1)/efi/DTS)/EFI/BOOT/grub.cfg.
  Booting `boot'
(hang forver)

or

FS0:\> \efi\BOOT\bootx64.efi
error: no such device: ((hd0,msdos1)/efi/BOOT)/EFI/BOOT/grub.cfg.
  Booting `boot'
(hang forver)

Screenshots

No response

Additional context

I'm trying to replace Alpine Linux in my workflow for firmware training.

Solutions you've tried

Use i440fx instead of q35. Maybe the issue is in wic image format?

pietrushnic commented 1 month ago

v1.2.19 ISO works fine:

qemu-system-x86_64 \
  -drive if=pflash,format=raw,readonly=on,file=Build/OvmfX64/DEBUG_GCC5/FV/OVMF_CODE.fd \
  -drive if=pflash,format=raw,file=Build/OvmfX64/DEBUG_GCC5/FV/OVMF_VARS.fd \
  -net none \
  -nographic \
  -chardev file,path=debug.log,id=ovmf-debug -device isa-debugcon,iobase=0x402,chardev=ovmf-debug \
  -device qemu-xhci,id=xhci -M q35 -m 2G \
  /home/user/dts-base-image-v1.2.19.iso

First, it goes to GRUB, where I modify the kernel boot option:

dts_1 2 19_qemu_boot

The boot process takes ~80s (quite long), but finally I get:

Dasharo Tools Suite 1.2.19 DasharoToolsSuite ttyS0

DasharoToolsSuite login: root (automatic login)

    ____             __                        ______            __        _____       _ __
   / __ \____ ______/ /_  ____ __________     /_  __/___  ____  / /____   / ___/__  __(_) /____
  / / / / __ `/ ___/ __ \/ __ `/ ___/ __ \     / / / __ \/ __ \/ / ___/   \__ \/ / / / / __/ _ \
 / /_/ / /_/ (__  ) / / / /_/ / /  / /_/ /    / / / /_/ / /_/ / (__  )   ___/ / /_/ / / /_/  __/
/_____/\__,_/____/_/ /_/\__,_/_/   \____/    /_/  \____/\____/_/____/   /____/\__,_/_/\__/\___/

  DTS version v1.2.19

  1) Dasharo HCL report - dump hardware information from this device
  2) Install Dasharo firmware
  4) Load your DES keys
  8) Start SSH server
  9) Shell
  10) Power off system
  11) Reboot system

Enter an option: 9

Entering shell, to leave type exit and press Enter or press LCtrl+D

Sometimes, when waiting too much in GRUB, I hit:

  Booting a command list

Kernel panic - not syncing: IO-APIC + timer doesn't work!  Boot with apic=debug and send a report.  Then try booting with the 'noapic' option.
CPU: 0 PID: 0 Comm: swapper/0 Not tainted 5.15.36-yocto-standard #1
Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS unknown 02/02/2022
Call Trace:
 <TASK>
 dump_stack_lvl+0x38/0x49
 dump_stack+0x10/0x12
 panic+0xf4/0x299
 setup_IO_APIC+0x811/0x877
 ? clear_IO_APIC+0x39/0x60
 apic_intr_mode_init+0x107/0x10f
 x86_late_time_init+0x24/0x35
 start_kernel+0x5b5/0x668
 x86_64_start_reservations+0x24/0x26
 x86_64_start_kernel+0x86/0x8a
 secondary_startup_64_no_verify+0xc2/0xcb
 </TASK>
---[ end Kernel panic - not syncing: IO-APIC + timer doesn't work!  Boot with apic=debug and send a report.  Then try booting with the 'noapic' option. ]---
m-iwanicki commented 1 month ago

@pietrushnic

Same if I use this file with OSFV from open-source-firmware-validation repository:

HDD_PATH=<path/to/>dts.img ./scripts/ci/qemu-run.sh graphic os

image

m-iwanicki commented 1 month ago

@pietrushnic managed to start DTS in QEMU that's run inside virtual machine.

  1. To get more verbose logs edit boot options:

    linux /bzImage root=PARTUUID=076c4a2a-02 rootwait console=ttyS0,115200n8 ignore_loglevel systemd.log_level=debug

    That way we can receive some information what's wrong

    systemd[1]: Set up TFD_TIMER_CANCEL_ON_SET timerfd.
systemd[1]: Failed to stat /etc/localtime, ignoring: No such file or directory
systemd[1]: /etc/localtime doesn't exist yet, watching /etc instead.
traps: systemd[1] trap invalid opcode ip:7f869d262c60 sp:7ffdf60c61f0 error:0 in libsystemd-core-255.so[7f869d1ce000+ff000]
systemd[1]: Caught <ILL> from PID -1658442656.
systemd[1]: Caught <ILL>, dumped core as pid 82.
systemd[1]: Freezing execution.

    I had similar errors when not using -enable-kvm or emulating wrong CPU.

  2. As -enable-kvm doesn't work inside your VM I decided to add -cpu Cascadelake-Server-v1 to your command from How to reproduce. There are some warnings from qemu so CPU used could be fine tuned, I choose first that's compatible with all ABI versions from https://qemu-project.gitlab.io/qemu/system/qemu-cpu-models.html

    qemu-system-x86_64: warning: TCG doesn't support requested feature: CPUID.01H:ECX.pcid [bit 17]
qemu-system-x86_64: warning: TCG doesn't support requested feature: CPUID.01H:ECX.x2apic [bit 21]
qemu-system-x86_64: warning: TCG doesn't support requested feature: CPUID.01H:ECX.tsc-deadline [bit 24]
qemu-system-x86_64: warning: TCG doesn't support requested feature: CPUID.07H:EBX.hle [bit 4]
qemu-system-x86_64: warning: TCG doesn't support requested feature: CPUID.07H:EBX.invpcid [bit 10]
qemu-system-x86_64: warning: TCG doesn't support requested feature: CPUID.07H:EBX.rtm [bit 11]
qemu-system-x86_64: warning: TCG doesn't support requested feature: CPUID.07H:EBX.avx512f [bit 16]
qemu-system-x86_64: warning: TCG doesn't support requested feature: CPUID.07H:EBX.avx512dq [bit 17]
qemu-system-x86_64: warning: TCG doesn't support requested feature: CPUID.07H:EBX.avx512cd [bit 28]
qemu-system-x86_64: warning: TCG doesn't support requested feature: CPUID.07H:EBX.avx512bw [bit 30]
qemu-system-x86_64: warning: TCG doesn't support requested feature: CPUID.07H:EBX.avx512vl [bit 31]
qemu-system-x86_64: warning: TCG doesn't support requested feature: CPUID.07H:ECX.avx512vnni [bit 11]
qemu-system-x86_64: warning: TCG doesn't support requested feature: CPUID.07H:EDX.spec-ctrl [bit 26]
qemu-system-x86_64: warning: TCG doesn't support requested feature: CPUID.07H:EDX.ssbd [bit 31]
qemu-system-x86_64: warning: TCG doesn't support requested feature: CPUID.0DH:EAX.xsavec [bit 1]
BdsDxe: loading Boot0002 "EFI Internal Shell" from Fv(7CB8BDC9-F8EB-4F34-AAEA-3EE4AF6516A1)/FvFile(7C04A583-9E3E-4F1C-AD65-E05268D0B4D1)
BdsDxe: starting Boot0002 "EFI Internal Shell" from Fv(7CB8BDC9-F8EB-4F34-AAEA-3EE4AF6516A1)/FvFile(7C04A583-9E3E-4F1C-AD65-E05268D0B4D1)

  3. Managed to boot into DTS (used just released v2.0.0)

     Dasharo Tools Suite Script 2.0.0
(c) Dasharo <contact@dasharo.com>
Report issues at: https://github.com/Dasharo/dasharo-issues
*********************************************************
**                HARDWARE INFORMATION
*********************************************************
**    System Inf.: QEMU Standard PC (Q35 + ICH9, 2009)
** Baseboard Inf.: QEMU
**       CPU Inf.: pc-q35-8.2
**    RAM DIMM 0: Not Specified
*********************************************************
**                FIRMWARE INFORMATION
*********************************************************
**      BIOS Inf.: EDK II unknown
*********************************************************
**     1) Dasharo HCL report
**     2) Install Dasharo Firmware
**     3) Restore firmware from Dasharo HCL report
**     4) Load your DPP keys
*********************************************************
R to reboot  P to poweroff  S to enter shell
K to launch SSH server  L to enable sending DTS logs V to enable verbose mode
Enter an option:

  4. I noticed that DTS is quite slow (e.g. showing interface is laggy). One way to drastically increase performance is to enable nested hardware virtualization, that way it's possible to use -enable-kvm argument in QEMU instead of emualting CPU. You might need to run qemu with sudo.

    image

    This time there is only one warning

    qemu-system-x86_64: warning: host doesn't support requested feature: CPUID.80000001H:ECX.svm [bit 2]
pietrushnic commented 1 month ago

Thanks, I will verify that I already have a code that includes DTS: https://gitlab.com/opensecuritytraining/ost2-vm-image-builder/-/blob/arch4221-support/scripts/ost2.sh?ref_type=heads#L34

Testing it and adjusting it according to what you point to is a matter of testing it. The critical question is how nested virtualization will behave on various OSes that students may use. Also, please note I'm using Packer, not UI, so I have to figure out how to use vboxmanage modifyvm to enable that, but that is something to figure out.