search

linuxboot / heads

A minimal Linux that runs as a coreboot or LinuxBoot ROM payload to provide a secure, flexible boot environment for laptops, workstations and servers.
https://osresearch.net/
GNU General Public License v2.0
1.41k stars 185 forks source link

Ubuntu 24.04 LTS does not show the luks encryption prompt when bootet with heads #1641

Open nestire opened 5 months ago

nestire commented 5 months ago

After a successful install via USB Drive Ubuntu is not booting and hangs at the point of "Starting new Kernel". It reacts to CRTL+ALT+ENTF but will not continue to boot in any case . Tried also the "Unsafe Boot" Option. No error message is shown, also adding "-d" here did not bring up any message.

Test it with the Nitrokey Heads Release 2.4.1 and with the test build from #1640 on nitropad ns70/nv41/t430

tlaurion commented 5 months ago

@nestire thanks for reporting this.

1640 removed CONFIG_FB_VESA=y https://github.com/linuxboot/heads/pull/1640/files#diff-782b88c1e0e03988fb8336bd99c65310869be9f3c1e3a88a1be57bcd5ab7c4e8R1759 since CONFIG_FB_EFI=y is supposed to be enough to boot from coreboot initialized ligfxinit/GOP FB compatible with kernel efifb driver with sysfb compatible FB passed from coreboot (which enables early boot bootsplash).

I will diff nitrokey 2.4.1 release linux config with heads master and #1640 further more.

Meanwhile, can you post a screenshot of the kexec command line where the kexec leads to a black screen please? Is master permitting to boot into 24.04 BETA LTS in current state?

tlaurion commented 5 months ago

I will try to add kexec output to dmesg in that PR so that I can ask you to turn on DEBUG + TRACE configuration option and provide me logs to diagnose this better.

tlaurion commented 5 months ago

@nestire https://github.com/linuxboot/heads/pull/1640/commits/fa70cba7ec9df887b24992550c961dbbc708d1c1 was added under #1640

After flashing, go under configuration settings, activate DEBUG+Tracing option, save in flash (will flash config back to flash) and then try to boot up to last prompt asking you if you really want to boot, on which please say no. You will land in recovery shell.

put a supported FS based usb thumdrive, call 

mount-usb --mode rw 
cp /tpm/debug.log /media
umount /media

And drop the log here please.

tlaurion commented 5 months ago

Tried also the "Unsafe Boot" Option. No error message is shown, also adding "-d" here did not bring up any message.

https://github.com/linuxboot/heads/pull/1642#issuecomment-2065315700 ? @nestire : You meant "-f" on kexec-select-boot -b /boot -f?

There is no -d option under https://github.com/linuxboot/heads/blob/82179e4e9894c5cb503f85522ff57088bdd444dc/initrd/bin/kexec-boot#L14-L20

alexgithublab commented 5 months ago

@nestire fa70cba was added under #1640

After flashing, go under configuration settings, activate DEBUG+Tracing option, save in flash (will flash config back to flash) and then try to boot up to last prompt asking you if you really want to boot, on which please say no. You will land in recovery shell.

put a supported FS based usb thumdrive, call

mount-usb --mode rw 
cp /tpm/debug.log /media
umount /media

And drop the log here please.

@tlaurion

I flash this image

here are the logs :

TRACE: /etc/gui_functions(83): file_selector
TRACE: /bin/config-gui.sh(8): main
TRACE: /bin/kexec-select-boot(7): main
 *** WARNING: Hash of TPM2 primary key handle does not exist ***
 *** WARNING: Please rebuild the boot hash tree ***
DEBUG: Hash of TPM2 primary key handle does not exist under /boot/kexec_primhdl_hash.txt
TRACE: /etc/functions(325): check_config
TRACE: /etc/functions(687): scan_boot_options
TRACE: /bin/kexec-parse-boot(5): main
DEBUG: filedir= /boot/grub
DEBUG: bootdir= /boot
DEBUG: bootlen= 5
DEBUG: appenddir= /grub
DEBUG:  grub_entry : linux trimcmd prior of kernel/append parsing: linux /vmlinuz-6.8.0-22-generic root=/dev/mapper/ubuntu--vg-ubuntu--lv ro quiet splash $vt_handoff
DEBUG:  grub_entry: linux initrd= /initrd.img-6.8.0-22-generic
DEBUG:  grub_entry : linux trimcmd prior of kernel/append parsing: linux /vmlinuz-6.8.0-22-generic root=/dev/mapper/ubuntu--vg-ubuntu--lv ro quiet splash $vt_handoff
DEBUG:  grub_entry: linux initrd= /initrd.img-6.8.0-22-generic
DEBUG:  grub_entry : linux trimcmd prior of kernel/append parsing: linux /vmlinuz-6.8.0-22-generic root=/dev/mapper/ubuntu--vg-ubuntu--lv ro recovery nomodeset dis_ucode_ldr
DEBUG:  grub_entry: linux initrd= /initrd.img-6.8.0-22-generic
DEBUG:  grub_entry : linux trimcmd prior of kernel/append parsing: linux /memtest86+x64.bin
DEBUG:  grub_entry : linux trimcmd prior of kernel/append parsing: linux /memtest86+x64.bin console=ttyS0,115200
TRACE: /bin/kexec-boot(7): main
DEBUG: kexectype= elf
DEBUG: restval=
DEBUG: filepath= /boot/vmlinuz-6.8.0-22-generic
DEBUG: kexeccmd= kexec -d -l /boot/vmlinuz-6.8.0-22-generic
DEBUG: PATH: /sbin:/bin:/usr/sbin:/usr/bin:/usr/local/bin
DEBUG: Executing command with cmd: kexec -d -l /boot/vmlinuz-6.8.0-22-generic --initrd=/boot/initrd.img-6.8.0-22-generic --append="root=/dev/mapper/ubuntu--vg-ubuntu--lv ro vt.default_red=0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff"
DEBUG: Command output: Try gzip decompression.
kernel: 0x7ff6713e9020 kernel_size: 0xe38988
MEMORY RANGES
0000000000000000-0000000000000fff 1
0000000000001000-000000000009ffff 0
00000000000a0000-00000000000fffff 1
0000000000100000-0000000076994fff 0
0000000076995000-00000000803fffff 1
00000000c0000000-00000000cfffffff 1
00000000f8000000-00000000f9ffffff 1
00000000fb000000-00000000fb000fff 1
00000000fc800000-00000000fe7fffff 1
00000000feb00000-00000000feb7ffff 1
00000000fec00000-00000000fecfffff 1
00000000fed40000-00000000fed6ffff 1
00000000fed80000-00000000fed87fff 1
00000000fed90000-00000000fed92fff 1
00000000feda0000-00000000feda1fff 1
00000000fedc0000-00000000feddffff 1
00000000ff000000-00000000ffffffff 1
0000000100000000-000000087fbfffff 0
sym: sha256_starts info: 12 other: 00 shndx: 1 value: 1120 size: 48
sym: sha256_starts value: 87fbf8120 addr: 87fbf7002
R_X86_64_64
sym: sha256_update info: 12 other: 00 shndx: 1 value: 3cb0 size: 19
sym: sha256_update value: 87fbfacb0 addr: 87fbf700f
R_X86_64_64
sym: sha256_regions info: 11 other: 00 shndx: 8 value: 40 size: 100
sym: sha256_regions value: 87fbfc040 addr: 87fbf701a
R_X86_64_64
sym: sha256_regions info: 11 other: 00 shndx: 8 value: 40 size: 100
sym: sha256_regions value: 87fbfc140 addr: 87fbf7044
R_X86_64_64
sym: sha256_finish info: 12 other: 00 shndx: 1 value: 3cd0 size: 15e
sym: sha256_finish value: 87fbfacd0 addr: 87fbf705b
R_X86_64_64
sym: sha256_digest info: 11 other: 00 shndx: 8 value: 20 size: 20
sym: sha256_digest value: 87fbfc020 addr: 87fbf706f
R_X86_64_64
sym:     memcmp info: 12 other: 00 shndx: 1 value: 599 size: 21
sym: memcmp value: 87fbf7599 addr: 87fbf7079
R_X86_64_64
sym: .rodata.str1.1 info: 03 other: 00 shndx: 5 value: 0 size: 0
sym: .rodata.str1.1 value: 87fbfaf40 addr: 87fbf708d
R_X86_64_64
sym:     printf info: 12 other: 00 shndx: 1 value: 4ba size: a0
sym: printf value: 87fbf74ba addr: 87fbf709b
R_X86_64_64
sym: .rodata.str1.1 info: 03 other: 00 shndx: 5 value: 0 size: 0
sym: .rodata.str1.1 value: 87fbfaf70 addr: 87fbf70a5
R_X86_64_64
sym: .rodata.str1.1 info: 03 other: 00 shndx: 5 value: 0 size: 0
sym: .rodata.str1.1 value: 87fbfaf60 addr: 87fbf70b3
R_X86_64_64
sym: .rodata.str1.1 info: 03 other: 00 shndx: 5 value: 0 size: 0
sym: .rodata.str1.1 value: 87fbfaf76 addr: 87fbf70d3
R_X86_64_64
sym: sha256_digest info: 11 other: 00 shndx: 8 value: 20 size: 20
sym: sha256_digest value: 87fbfc020 addr: 87fbf70e1
R_X86_64_64
sym: .rodata.str1.1 info: 03 other: 00 shndx: 5 value: 0 size: 0
sym: .rodata.str1.1 value: 87fbfaf78 addr: 87fbf70ef
R_X86_64_64
sym: .rodata.str1.1 info: 03 other: 00 shndx: 5 value: 0 size: 0
sym: .rodata.str1.1 value: 87fbfaf70 addr: 87fbf7105
R_X86_64_64
sym: .rodata.str1.1 info: 03 other: 00 shndx: 5 value: 0 size: 0
sym: .rodata.str1.1 value: 87fbfaf76 addr: 87fbf7117
R_X86_64_64
sym: .rodata.str1.1 info: 03 other: 00 shndx: 5 value: 0 size: 0
sym: .rodata.str1.1 value: 87fbfaf88 addr: 87fbf7136
R_X86_64_64
sym:     printf info: 12 other: 00 shndx: 1 value: 4ba size: a0
sym: printf value: 87fbf74ba addr: 87fbf7142
R_X86_64_64
sym: setup_arch info: 12 other: 00 shndx: 1 value: 762 size: 56
sym: setup_arch value: 87fbf7762 addr: 87fbf714f
R_X86_64_64
sym: skip_checks info: 11 other: 00 shndx: 8 value: 0 size: 4
sym: skip_checks value: 87fbfc000 addr: 87fbf715b
R_X86_64_64
sym: verify_sha256_digest info: 12 other: 00 shndx: 1 value: 0 size: 134
sym: verify_sha256_digest value: 87fbf7000 addr: 87fbf716a
R_X86_64_64
sym: post_verification_setup_arch info: 12 other: 00 shndx: 1 value: 7f0 size: 58
sym: post_verification_setup_arch value: 87fbf77f0 addr: 87fbf717c
R_X86_64_64
sym:    putchar info: 12 other: 00 shndx: 1 value: d8e size: 124
sym: putchar value: 87fbf7d8e addr: 87fbf718f
R_X86_64_64
sym: .rodata.str1.1 info: 03 other: 00 shndx: 5 value: 0 size: 0
sym: .rodata.str1.1 value: 87fbfaf9a addr: 87fbf72d1
R_X86_64_64
sym:   vsprintf info: 12 other: 00 shndx: 1 value: 187 size: 29d
sym: vsprintf value: 87fbf7187 addr: 87fbf74a0
R_X86_64_64
sym:   vsprintf info: 12 other: 00 shndx: 1 value: 187 size: 29d
sym: vsprintf value: 87fbf7187 addr: 87fbf7540
R_X86_64_64
sym:    entry32 info: 10 other: 00 shndx: 1 value: 5c0 size: 0
sym: entry32 value: 87fbf75bc addr: 87fbf75cd
R_X86_64_PC32
sym:    entry32 info: 10 other: 00 shndx: 1 value: 5c0 size: 0
sym: entry32 value: 87fbf75bc addr: 87fbf75e2
R_X86_64_PC32
sym:      .data info: 03 other: 00 shndx: 8 value: 0 size: 0
sym: .data value: 87fbfc15c addr: 87fbf75fd
R_X86_64_PC32
sym:      .data info: 03 other: 00 shndx: 8 value: 0 size: 0
sym: .data value: 87fbfc13c addr: 87fbf7604
R_X86_64_PC32
sym:    .rodata info: 03 other: 00 shndx: 3 value: 0 size: 0
sym: .rodata value: 87fbfae54 addr: 87fbf760a
R_X86_64_PC32
sym:      .data info: 03 other: 00 shndx: 8 value: 0 size: 0
sym: .data value: 87fbfc1ac addr: 87fbf7610
R_X86_64_PC32
sym:    .rodata info: 03 other: 00 shndx: 3 value: 0 size: 0
sym: .rodata value: 87fbfae2c addr: 87fbf7616
R_X86_64_PC32
sym:    .rodata info: 03 other: 00 shndx: 3 value: 0 size: 0
sym: .rodata value: 87fbfaec1 addr: 87fbf7649
R_X86_64_PC32
sym:    .rodata info: 03 other: 00 shndx: 3 value: 0 size: 0
sym: .rodata value: 87fbfaed0 addr: 87fbf7650
R_X86_64_PC32
sym:    .rodata info: 03 other: 00 shndx: 3 value: 0 size: 0
sym: .rodata value: 87fbfaedb addr: 87fbf7657
R_X86_64_PC32
sym:    .rodata info: 03 other: 00 shndx: 3 value: 0 size: 0
sym: .rodata value: 87fbfaee6 addr: 87fbf765e
R_X86_64_PC32
sym:    .rodata info: 03 other: 00 shndx: 3 value: 0 size: 0
sym: .rodata value: 87fbfaef1 addr: 87fbf7665
R_X86_64_PC32
sym:    .rodata info: 03 other: 00 shndx: 3 value: 0 size: 0
sym: .rodata value: 87fbfaefc addr: 87fbf766c
R_X86_64_PC32
sym:    .rodata info: 03 other: 00 shndx: 3 value: 0 size: 0
sym: .rodata value: 87fbfaf07 addr: 87fbf7673
R_X86_64_PC32
sym:    .rodata info: 03 other: 00 shndx: 3 value: 0 size: 0
sym: .rodata value: 87fbfaef6 addr: 87fbf767a
R_X86_64_PC32
sym:      .data info: 03 other: 00 shndx: 8 value: 0 size: 0
sym: .data value: 87fbfc271 addr: 87fbf7681
R_X86_64_PC32
sym:    .rodata info: 03 other: 00 shndx: 3 value: 0 size: 0
sym: .rodata value: 87fbfaeec addr: 87fbf7693
R_X86_64_PC32
sym:    .rodata info: 03 other: 00 shndx: 3 value: 0 size: 0
sym: .rodata value: 87fbfaf1c addr: 87fbf76a9
R_X86_64_PC32
sym:    .rodata info: 03 other: 00 shndx: 3 value: 0 size: 0
sym: .rodata value: 87fbfae5c addr: 87fbf76bc
R_X86_64_PC32
sym:    .rodata info: 03 other: 00 shndx: 3 value: 0 size: 0
sym: .rodata value: 87fbfae64 addr: 87fbf76c3
R_X86_64_PC32
sym:    .rodata info: 03 other: 00 shndx: 3 value: 0 size: 0
sym: .rodata value: 87fbfae6c addr: 87fbf76ca
R_X86_64_PC32
sym:    .rodata info: 03 other: 00 shndx: 3 value: 0 size: 0
sym: .rodata value: 87fbfae74 addr: 87fbf76d1
R_X86_64_PC32
sym:    .rodata info: 03 other: 00 shndx: 3 value: 0 size: 0
sym: .rodata value: 87fbfae7c addr: 87fbf76d8
R_X86_64_PC32
sym:    .rodata info: 03 other: 00 shndx: 3 value: 0 size: 0
sym: .rodata value: 87fbfae84 addr: 87fbf76df
R_X86_64_PC32
sym:    .rodata info: 03 other: 00 shndx: 3 value: 0 size: 0
sym: .rodata value: 87fbfae8c addr: 87fbf76e6
R_X86_64_PC32
sym:    .rodata info: 03 other: 00 shndx: 3 value: 0 size: 0
sym: .rodata value: 87fbfae94 addr: 87fbf76ed
R_X86_64_PC32
sym:    .rodata info: 03 other: 00 shndx: 3 value: 0 size: 0
sym: .rodata value: 87fbfae9c addr: 87fbf76f4
R_X86_64_PC32
sym:    .rodata info: 03 other: 00 shndx: 3 value: 0 size: 0
sym: .rodata value: 87fbfaea4 addr: 87fbf76fb
R_X86_64_PC32
sym:    .rodata info: 03 other: 00 shndx: 3 value: 0 size: 0
sym: .rodata value: 87fbfaeac addr: 87fbf7702
R_X86_64_PC32
sym:    .rodata info: 03 other: 00 shndx: 3 value: 0 size: 0
sym: .rodata value: 87fbfaeb4 addr: 87fbf7709
R_X86_64_PC32
sym:    .rodata info: 03 other: 00 shndx: 3 value: 0 size: 0
sym: .rodata value: 87fbfaebc addr: 87fbf7710
R_X86_64_PC32
sym:    .rodata info: 03 other: 00 shndx: 3 value: 0 size: 0
sym: .rodata value: 87fbfaec4 addr: 87fbf7717
R_X86_64_PC32
sym:    .rodata info: 03 other: 00 shndx: 3 value: 0 size: 0
sym: .rodata value: 87fbfaecc addr: 87fbf771e
R_X86_64_PC32
sym:    .rodata info: 03 other: 00 shndx: 3 value: 0 size: 0
sym: .rodata value: 87fbfaed4 addr: 87fbf7725
R_X86_64_PC32
sym:    .rodata info: 03 other: 00 shndx: 3 value: 0 size: 0
sym: .rodata value: 87fbfaedc addr: 87fbf772b
R_X86_64_PC32
sym:    .rodata info: 03 other: 00 shndx: 3 value: 0 size: 0
sym: .rodata value: 87fbfaf1c addr: 87fbf7733
R_X86_64_PC32
sym: jump_back_entry info: 11 other: 00 shndx: 8 value: 2008 size: 8
sym: jump_back_entry value: 87fbfe004 addr: 87fbf774d
R_X86_64_PC32
sym:       .bss info: 03 other: 00 shndx: a value: 0 size: 0
sym: .bss value: 87fbffffc addr: 87fbf7754
R_X86_64_PC32
sym:  purgatory info: 12 other: 00 shndx: 1 value: 134 size: 53
sym: purgatory value: 87fbf7130 addr: 87fbf7759
R_X86_64_PLT32
sym:    entry64 info: 10 other: 00 shndx: 1 value: 690 size: 0
sym: entry64 value: 87fbf768c addr: 87fbf775e
R_X86_64_PLT32
sym:  reset_vga info: 11 other: 00 shndx: 8 value: 2012 size: 1
sym: reset_vga value: 87fbfe012 addr: 87fbf7764
R_X86_64_64
sym: x86_reset_vga info: 12 other: 00 shndx: 1 value: eb2 size: 232
sym: x86_reset_vga value: 87fbf7eb2 addr: 87fbf7773
R_X86_64_64
sym: legacy_pic info: 11 other: 00 shndx: 8 value: 2011 size: 1
sym: legacy_pic value: 87fbfe011 addr: 87fbf7780
R_X86_64_64
sym: x86_setup_legacy_pic info: 12 other: 00 shndx: 1 value: 10e4 size: 35
sym: x86_setup_legacy_pic value: 87fbf80e4 addr: 87fbf778f
R_X86_64_64
sym: legacy_pic info: 11 other: 00 shndx: 8 value: 2011 size: 1
sym: legacy_pic value: 87fbfe011 addr: 87fbf779e
R_X86_64_64
sym: x86_setup_legacy_pic info: 12 other: 00 shndx: 1 value: 10e4 size: 35
sym: x86_setup_legacy_pic value: 87fbf80e4 addr: 87fbf77ad
R_X86_64_64
sym: cmdline_end info: 11 other: 00 shndx: 8 value: 2000 size: 8
sym: cmdline_end value: 87fbfe000 addr: 87fbf77ba
R_X86_64_64
sym: jump_back_entry info: 11 other: 00 shndx: 8 value: 2008 size: 8
sym: jump_back_entry value: 87fbfe008 addr: 87fbf77cc
R_X86_64_64
sym: .rodata.str1.1 info: 03 other: 00 shndx: 5 value: 0 size: 0
sym: .rodata.str1.1 value: 87fbfafab addr: 87fbf77db
R_X86_64_64
sym:    sprintf info: 12 other: 00 shndx: 1 value: 424 size: 96
sym: sprintf value: 87fbf7424 addr: 87fbf77e5
R_X86_64_64
sym: panic_kernel info: 11 other: 00 shndx: 8 value: 2010 size: 1
sym: panic_kernel value: 87fbfe010 addr: 87fbf77f2
R_X86_64_64
sym: crashdump_backup_memory info: 12 other: 00 shndx: 1 value: cb3 size: 3e
sym: crashdump_backup_memory value: 87fbf7cb3 addr: 87fbf7801
R_X86_64_64
sym: jump_back_entry info: 11 other: 00 shndx: 8 value: 2008 size: 8
sym: jump_back_entry value: 87fbfe008 addr: 87fbf780e
R_X86_64_64
sym: x86_setup_jump_back_entry info: 12 other: 00 shndx: 1 value: 7b8 size: 38
sym: x86_setup_jump_back_entry value: 87fbf77b8 addr: 87fbf781e
R_X86_64_64
sym: jump_back_entry info: 11 other: 00 shndx: 8 value: 2008 size: 8
sym: jump_back_entry value: 87fbfe008 addr: 87fbf782d
R_X86_64_64
sym: x86_setup_jump_back_entry info: 12 other: 00 shndx: 1 value: 7b8 size: 38
sym: x86_setup_jump_back_entry value: 87fbf77b8 addr: 87fbf783d
R_X86_64_64
sym:      .data info: 03 other: 00 shndx: 8 value: 0 size: 0
sym: .data value: 87fbfe047 addr: 87fbf7867
R_X86_64_PC32
sym:      .data info: 03 other: 00 shndx: 8 value: 0 size: 0
sym: .data value: 87fbfe04f addr: 87fbf786d
R_X86_64_PC32
sym:      .data info: 03 other: 00 shndx: 8 value: 0 size: 0
sym: .data value: 87fbfe075 addr: 87fbf788d
R_X86_64_PC32
sym:      .data info: 03 other: 00 shndx: 8 value: 0 size: 0
sym: .data value: 87fbfe07f addr: 87fbf7893
R_X86_64_PC32
sym:      .data info: 03 other: 00 shndx: 8 value: 0 size: 0
sym: .data value: 87fbfe089 addr: 87fbf7899
R_X86_64_PC32
sym:      .data info: 03 other: 00 shndx: 8 value: 0 size: 0
sym: .data value: 87fbfe093 addr: 87fbf789f
R_X86_64_PC32
sym:      .data info: 03 other: 00 shndx: 8 value: 0 size: 0
sym: .data value: 87fbfe076 addr: 87fbf78a6
R_X86_64_PC32
sym:      .data info: 03 other: 00 shndx: 8 value: 0 size: 0
sym: .data value: 87fbfe08d addr: 87fbf78ad
R_X86_64_PC32
sym:      .data info: 03 other: 00 shndx: 8 value: 0 size: 0
sym: .data value: 87fbfe06d addr: 87fbf7977
R_X86_64_PC32
sym:      .data info: 03 other: 00 shndx: 8 value: 0 size: 0
sym: .data value: 87fbfe075 addr: 87fbf797d
R_X86_64_PC32
sym:      .data info: 03 other: 00 shndx: 8 value: 0 size: 0
sym: .data value: 87fbfe09b addr: 87fbf799d
R_X86_64_PC32
sym:      .data info: 03 other: 00 shndx: 8 value: 0 size: 0
sym: .data value: 87fbfe0a5 addr: 87fbf79a3
R_X86_64_PC32
sym:      .data info: 03 other: 00 shndx: 8 value: 0 size: 0
sym: .data value: 87fbfe0af addr: 87fbf79a9
R_X86_64_PC32
sym:      .data info: 03 other: 00 shndx: 8 value: 0 size: 0
sym: .data value: 87fbfe0b9 addr: 87fbf79af
R_X86_64_PC32
sym:      .data info: 03 other: 00 shndx: 8 value: 0 size: 0
sym: .data value: 87fbfe0e2 addr: 87fbf79f2
R_X86_64_PC32
sym:      .data info: 03 other: 00 shndx: 8 value: 0 size: 0
sym: .data value: 87fbfe12b addr: 87fbf7a35
R_X86_64_PC32
sym: backup_src_size info: 11 other: 00 shndx: 8 value: 2070 size: 8
sym: backup_src_size value: 87fbfe070 addr: 87fbf7cb5
R_X86_64_64
sym: backup_start info: 11 other: 00 shndx: 8 value: 2080 size: 8
sym: backup_start value: 87fbfe080 addr: 87fbf7cc7
R_X86_64_64
sym: backup_src_start info: 11 other: 00 shndx: 8 value: 2078 size: 8
sym: backup_src_start value: 87fbfe078 addr: 87fbf7cd9
R_X86_64_64
sym:     memcpy info: 12 other: 00 shndx: 1 value: 581 size: 18
sym: memcpy value: 87fbf7581 addr: 87fbf7ce6
R_X86_64_64
sym: serial_base info: 11 other: 00 shndx: 8 value: 2090 size: 2
sym: serial_base value: 87fbfe090 addr: 87fbf7cf3
R_X86_64_64
sym:      .data info: 03 other: 00 shndx: 8 value: 0 size: 0
sym: .data value: 87fbfe088 addr: 87fbf7d00
R_X86_64_64
sym: serial_baud info: 11 other: 00 shndx: 8 value: 208c size: 4
sym: serial_baud value: 87fbfe08c addr: 87fbf7d3a
R_X86_64_64
sym:      .data info: 03 other: 00 shndx: 8 value: 0 size: 0
sym: .data value: 87fbfe088 addr: 87fbf7d69
R_X86_64_64
sym: console_vga info: 11 other: 00 shndx: 8 value: 2093 size: 1
sym: console_vga value: 87fbfe093 addr: 87fbf7d90
R_X86_64_64
sym:      .data info: 03 other: 00 shndx: 8 value: 0 size: 0
sym: .data value: 87fbfe098 addr: 87fbf7da4
R_X86_64_64
sym:      .data info: 03 other: 00 shndx: 8 value: 0 size: 0
sym: .data value: 87fbfe098 addr: 87fbf7e09
R_X86_64_64
sym:      .data info: 03 other: 00 shndx: 8 value: 0 size: 0
sym: .data value: 87fbfe094 addr: 87fbf7e1e
R_X86_64_64
sym:      .data info: 03 other: 00 shndx: 8 value: 0 size: 0
sym: .data value: 87fbfe094 addr: 87fbf7e37
R_X86_64_64
sym:      .data info: 03 other: 00 shndx: 8 value: 0 size: 0
sym: .data value: 87fbfe094 addr: 87fbf7e6d
R_X86_64_64
sym: console_serial info: 11 other: 00 shndx: 8 value: 2092 size: 1
sym: console_serial value: 87fbfe092 addr: 87fbf7e81
R_X86_64_64
sym:      .text info: 03 other: 00 shndx: 1 value: 0 size: 0
sym: .text value: 87fbf7cf1 addr: 87fbf7e90
R_X86_64_64
sym: sha256_process info: 12 other: 00 shndx: 1 value: 1170 size: 2a25
sym: sha256_process value: 87fbf8170 addr: 87fbfabef
R_X86_64_64
sym:     memcpy info: 12 other: 00 shndx: 1 value: 581 size: 18
sym: memcpy value: 87fbf7581 addr: 87fbfac53
R_X86_64_64
sym: sha256_process info: 12 other: 00 shndx: 1 value: 1170 size: 2a25
sym: sha256_process value: 87fbf8170 addr: 87fbfac65
R_X86_64_64
sym:     memcpy info: 12 other: 00 shndx: 1 value: 581 size: 18
sym: memcpy value: 87fbf7581 addr: 87fbfac93
R_X86_64_64
sym:      .text info: 03 other: 00 shndx: 1 value: 0 size: 0
sym: .text value: 87fbfaba0 addr: 87fbfacb7
R_X86_64_64
sym:      .text info: 03 other: 00 shndx: 1 value: 0 size: 0
sym: .text value: 87fbfaba0 addr: 87fbfad1a
R_X86_64_64
sym:      .data info: 03 other: 00 shndx: 8 value: 0 size: 0
sym: .data value: 87fbfe0a0 addr: 87fbfad27
R_X86_64_64
sym:    entry16 info: 10 other: 00 shndx: 1 value: 850 size: 0
sym: entry16 value: 87fbf7850 addr: 87fbfae58
R_X86_64_64
sym:    entry32 info: 10 other: 00 shndx: 1 value: 5c0 size: 0
sym: entry32 value: 87fbf75c0 addr: 87fbfaee0
R_X86_64_64
sym:    .rodata info: 03 other: 00 shndx: 3 value: 0 size: 0
sym: .rodata value: 87fbfaef0 addr: 87fbfaef2
R_X86_64_64
sym:    .rodata info: 03 other: 00 shndx: 3 value: 0 size: 0
sym: .rodata value: 87fbfaf20 addr: 87fbfaf22
R_X86_64_64
sym:      .text info: 03 other: 00 shndx: 1 value: 0 size: 0
sym: .text value: 87fbf7000 addr: 87fbfafe8
R_X86_64_PC32
sym:      .text info: 03 other: 00 shndx: 1 value: 0 size: 0
sym: .text value: 87fbf7134 addr: 87fbfb01c
R_X86_64_PC32
sym:      .text info: 03 other: 00 shndx: 1 value: 0 size: 0
sym: .text value: 87fbf7187 addr: 87fbfb050
R_X86_64_PC32
sym:      .text info: 03 other: 00 shndx: 1 value: 0 size: 0
sym: .text value: 87fbf7424 addr: 87fbfb09c
R_X86_64_PC32
sym:      .text info: 03 other: 00 shndx: 1 value: 0 size: 0
sym: .text value: 87fbf74ba addr: 87fbfb0b8
R_X86_64_PC32
sym:      .text info: 03 other: 00 shndx: 1 value: 0 size: 0
sym: .text value: 87fbf755a addr: 87fbfb0f0
R_X86_64_PC32
sym:      .text info: 03 other: 00 shndx: 1 value: 0 size: 0
sym: .text value: 87fbf756d addr: 87fbfb104
R_X86_64_PC32
sym:      .text info: 03 other: 00 shndx: 1 value: 0 size: 0
sym: .text value: 87fbf7581 addr: 87fbfb118
R_X86_64_PC32
sym:      .text info: 03 other: 00 shndx: 1 value: 0 size: 0
sym: .text value: 87fbf7599 addr: 87fbfb12c
R_X86_64_PC32
sym:      .text info: 03 other: 00 shndx: 1 value: 0 size: 0
sym: .text value: 87fbf7762 addr: 87fbfb158
R_X86_64_PC32
sym:      .text info: 03 other: 00 shndx: 1 value: 0 size: 0
sym: .text value: 87fbf77b8 addr: 87fbfb178
R_X86_64_PC32
sym:      .text info: 03 other: 00 shndx: 1 value: 0 size: 0
sym: .text value: 87fbf77f0 addr: 87fbfb18c
R_X86_64_PC32
sym:      .text info: 03 other: 00 shndx: 1 value: 0 size: 0
sym: .text value: 87fbf7cb3 addr: 87fbfb1c8
R_X86_64_PC32
sym:      .text info: 03 other: 00 shndx: 1 value: 0 size: 0
sym: .text value: 87fbf7cf1 addr: 87fbfb1f8
R_X86_64_PC32
sym:      .text info: 03 other: 00 shndx: 1 value: 0 size: 0
sym: .text value: 87fbf7d8e addr: 87fbfb20c
R_X86_64_PC32
sym:      .text info: 03 other: 00 shndx: 1 value: 0 size: 0
sym: .text value: 87fbf7eb2 addr: 87fbfb248
R_X86_64_PC32
sym:      .text info: 03 other: 00 shndx: 1 value: 0 size: 0
sym: .text value: 87fbf80e4 addr: 87fbfb278
R_X86_64_PC32
sym:      .text info: 03 other: 00 shndx: 1 value: 0 size: 0
sym: .text value: 87fbf8120 addr: 87fbfb2a8
R_X86_64_PC32
sym:      .text info: 03 other: 00 shndx: 1 value: 0 size: 0
sym: .text value: 87fbf8170 addr: 87fbfb2bc
R_X86_64_PC32
sym:      .text info: 03 other: 00 shndx: 1 value: 0 size: 0
sym: .text value: 87fbfaba0 addr: 87fbfb308
R_X86_64_PC32
sym:      .text info: 03 other: 00 shndx: 1 value: 0 size: 0
sym: .text value: 87fbfacb0 addr: 87fbfb36c
R_X86_64_PC32
sym:      .text info: 03 other: 00 shndx: 1 value: 0 size: 0
sym: .text value: 87fbfacd0 addr: 87fbfb380
R_X86_64_PC32
sym:      .data info: 03 other: 00 shndx: 8 value: 0 size: 0
sym: .data value: 87fbfc140 addr: 87fbfc142
R_X86_64_64
Loaded purgatory at addr 0x87fbf7000
Loaded real_mode_data and command line at 0x87fbf1000
kernel init_size 0x3e8f000
Loaded 64bit kernel at 0x87bc00000
Loaded initrd at 0x8777c5000 size 0x443a84d
setup_linux_vesafb: Found driver EFI VGA, providing VIDEO_TYPE_EFI
E820 memmap:
0000000000000000-0000000000000fff 2
0000000000001000-000000000009ffff 1
00000000000a0000-00000000000fffff 2
0000000000100000-0000000076994fff 1
0000000076995000-00000000803fffff 2
00000000c0000000-00000000cfffffff 2
00000000f8000000-00000000f9ffffff 2
00000000fb000000-00000000fb000fff 2
00000000fc800000-00000000fe7fffff 2
00000000feb00000-00000000feb7ffff 2
00000000fec00000-00000000fecfffff 2
00000000fed40000-00000000fed6ffff 2
00000000fed80000-00000000fed87fff 2
00000000fed90000-00000000fed92fff 2
00000000feda0000-00000000feda1fff 2
00000000fedc0000-00000000feddffff 2
00000000ff000000-00000000ffffffff 2
0000000100000000-000000087fbfffff 1
/sys/firmware/edd does not exist.
kexec_load: entry = 0x87fbf7730 flags = 0x3e0000
nr_segments = 5
segment[0].buf   = 0x6fbf20
segment[0].bufsz = 0x30
segment[0].mem   = 0x100000
segment[0].memsz = 0x1000
segment[1].buf   = 0x7ff66cfae020
segment[1].bufsz = 0x443a84d
segment[1].mem   = 0x8777c5000
segment[1].memsz = 0x443b000
segment[2].buf   = 0x7ff6713ee020
segment[2].bufsz = 0xe33988
segment[2].mem   = 0x87bc00000
segment[2].memsz = 0x3e8f000
segment[3].buf   = 0x1263120
segment[3].bufsz = 0x5068
segment[3].mem   = 0x87fbf1000
segment[3].memsz = 0x6000
segment[4].buf   = 0x125c020
segment[4].bufsz = 0x70e0
segment[4].mem   = 0x87fbf7000
segment[4].memsz = 0x9000
DEBUG: Command exited with status: 0
DEBUG: kexeccmd= kexec -d -l /boot/vmlinuz-6.8.0-22-generic --initrd=/boot/initrd.img-6.8.0-22-generic --append="root=/dev/mapper/ubuntu--vg-ubuntu--lv ro    vt.default_red=0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff"
 !!! ERROR: Boot aborted !!!
 !!! ERROR: !!! Failed to boot w/ options: Ubuntu|elf|kernel /vmlinuz-6.8.0-22-generic|initrd /initrd.img-6.8.0-22-generic|append root=/dev/mapper/ubuntu--vg-ubuntu--lv ro quiet splash !!!
DEBUG: Extending TPM PCR 4 for recovery shell access
TRACE: /bin/tpmr(32): main
TRACE: /bin/tpmr(232): tpm2_extend
DEBUG: PATH: /sbin:/bin:/usr/sbin:/usr/bin:/usr/local/bin
DEBUG: Executing command with cmd: tpm2 pcrread sha256:4
DEBUG: Command output:   sha256:
4 : 0x22EB50C62F72A3178CEFEE9149954615AA1E47C67801AB2AFF5A1A1B10FA1A1B
DEBUG: Command exited with status: 0
TRACE: /bin/mount-usb(6): main
DEBUG: Parameters: --mode=rw, --device=empty, --mountpoint=/media, --pass=
TRACE: Under /etc/ash_functions:enable_usb
TRACE: /sbin/insmod(9): main
DEBUG: /lib/modules/ehci-hcd.ko: already loaded
TRACE: /sbin/insmod(9): main
DEBUG: /lib/modules/ehci-pci.ko: already loaded
TRACE: /sbin/insmod(9): main
DEBUG: /lib/modules/xhci-hcd.ko: already loaded
TRACE: /sbin/insmod(9): main
DEBUG: /lib/modules/xhci-pci.ko: already loaded
TRACE: /etc/functions(153): enable_usb_storage
TRACE: /sbin/insmod(9): main
DEBUG: Extending TPM PCR 5 with /lib/modules/usb-storage.ko prior of usage
TRACE: /bin/tpmr(32): main
TRACE: /bin/tpmr(232): tpm2_extend
DEBUG: PATH: /sbin:/bin:/usr/sbin:/usr/bin:/usr/local/bin
DEBUG: Executing command with cmd: tpm2 pcrread sha256:5
DEBUG: Command output:   sha256:
5 : 0x77CF964C10C4BB0EC36BB9689379474E8CB75530D0C2738E5B2B2D9CA70C23E3
DEBUG: Command exited with status: 0
DEBUG: Loading /lib/modules/usb-storage.ko with busybox insmod
TRACE: /etc/functions(190): list_usb_storage
DEBUG: Listing USB storage devices (including partitions)
TRACE: /etc/functions(190): list_usb_storage
DEBUG: Listing USB storage devices (including partitions)
TRACE: /etc/functions(190): list_usb_storage
DEBUG: Listing USB storage devices (including partitions)
DEBUG: USB storage device of size greater then 0: /sys/block/sda
DEBUG: USB storage device with partition table: /dev/sda
DEBUG: Cleaning /media directory
TRACE: /etc/functions(190): list_usb_storage
DEBUG: Listing USB storage devices (including partitions)
DEBUG: USB storage device of size greater then 0: /sys/block/sda
DEBUG: USB storage device with partition table: /dev/sda
DEBUG: Checking if /dev/sda1 is a LUKS device/partition
DEBUG: Selected USB partition is not a LUKS device, continuing...
DEBUG: Mounting /dev/sda1 as read-write
tlaurion commented 5 months ago

@alexgithublab

setup_linux_vesafb: Found driver EFI VGA, providing VIDEO_TYPE_EFI

So FB from coreboot, discovered by Heads Kernel as efifb and passed down to next kernel in kexec -l not in cause.

/boot/vmlinuz-6.8.0-22-generic --initrd=/boot/initrd.img-6.8.0-22-generic --append="root=/dev/mapper/ubuntu--vg-ubuntu--lv ro vt.default_red=0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff"

All seems good. Will have to replicate.

tlaurion commented 5 months ago

Hypothesises:

https://github.com/linuxboot/heads/pull/1640#discussion_r1570881191 https://github.com/linuxboot/heads/pull/1640#discussion_r1570866698

tlaurion commented 5 months ago

As referred from other issue, that version of Ubuntu can be booted on x230. Not with a TPM Disk Unlock Key (DUK) from opened bug, but with Disk Recovery Key passphrase from Ubuntu initramfs to unlock LUKS container.

This means that the "Heads black screen after kexec" stroked again.

Will follow past hypothesis. Normally, Ubuntu, and all recent OSes provides efifb/simplefb instead of other legacy options, which you were able to get into at the installer.

The the installer figures out what is available, loads other modules and keep trace of loaded modules that functioned to construct the initramfs to be deployed to /boot, which heads launches alongside of the kernel in its kexec callfor whatever reason here, if the installer worked, the troubleshooting path will be to see what gets loaded and what is included in the constructed initramfs that is then attempted to boot, and fails to drive the display post kexec call.

nestire commented 5 months ago

lsmod_installer.txt dmesg_installer.txt

modules.builtin_initrd_finished_install.txt

So this might also be a ubuntu installer issue here?

alexgithublab commented 5 months ago

@tlaurion @nestire

ubuntu install just worked for me. I used the same ISO image than @nestire but when launching the installer it ask me to update it with a new version.

So It seems that they fixed it.

Also just before booting on it in Heads I got something similar to this :

/boot/vmlinuz-6.8.0-22-generic --initrd=/boot/initrd.img-6.8.0-22-generic --append="root=/dev/mapper/ubuntu--vg-ubuntu--lv ro vt.default_red=0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff"

tlaurion commented 5 months ago

@tlaurion @nestire

ubuntu install just worked for me. I used the same ISO image than @nestire but when launching the installer it ask me to update it with a new version.

Did you mean same rom as @nestire ? Latest from PR produced rom image on Circleci?

So It seems that they fixed it.

Or you mean latest non beta iso?

Also just before booting on it in Heads I got something similar to this :

/boot/vmlinuz-6.8.0-22-generic --initrd=/boot/initrd.img-6.8.0-22-generic --append="root=/dev/mapper/ubuntu--vg-ubuntu--lv ro vt.default_red=0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff"

The vt kernel command line appended is the result of booting in unsafe mode and it to change console background color to red.

alexgithublab commented 5 months ago

@tlaurion I mean the same installation image of Ubuntu

alexgithublab commented 5 months ago

My bad, ubuntu 24 is still not working..

Yesterday I figured out that Installing without LUKS encryption will make ubuntu bootable but OEM factory reset will not work + errors with dongle

tlaurion commented 5 months ago

Ok for further PR to fix this issue:

From https://github.com/linuxboot/heads/pull/1640#issuecomment-2067723954

So only relevant changes under https://github.com/linuxboot/heads/compare/521c0b039ea95c9133566944d2e4bc29a9772507..16f1d07867ddca9a5feee1f9541e2a7cc52d3b4a outside of rebasing on master and removing irrelevant changes here:

  • shared linux config being exactly the same as librems now, good or bad to be determined
  • coreboot config changed to match as close as possible librem_11 here which is also GOP GB driven.

OP and comments here to be hidden/review resolved since splitted under #1642 and merged in master yesterday to ease debugging forward.

tlaurion commented 5 months ago

lsmod_installer.txt dmesg_installer.txt

modules.builtin_initrd_finished_install.txt

So this might also be a ubuntu installer issue here?

@nestire Interesting. Will check your logs and leave proper debugging trace

tlaurion commented 5 months ago

lsmod_installer.txt dmesg_installer.txt modules.builtin_initrd_finished_install.txt So this might also be a ubuntu installer issue here?

@nestire Interesting. Will check your logs and leave proper debugging trace

user@heads-tests-deb12:~/Downloads$ grep -e drm -e fb -e efi -e xe dmesg_installer.txt lsmod_installer.txt modules.builtin_initrd_finished_install.txt 
dmesg_installer.txt:[    0.000000] BIOS-e820: [mem 0x00000000fb000000-0x00000000fb000fff] reserved
dmesg_installer.txt:[    0.000000] BIOS-e820: [mem 0x0000000100000000-0x000000087fbfffff] usable
dmesg_installer.txt:[    0.000000] NX (Execute Disable) protection: active
dmesg_installer.txt:[    0.000000] reserve setup_data: [mem 0x00000000fb000000-0x00000000fb000fff] reserved
dmesg_installer.txt:[    0.000000] reserve setup_data: [mem 0x0000000100000000-0x000000087fbfffff] usable
dmesg_installer.txt:[    0.000014] MTRR map: 6 entries (3 fixed + 3 variable; max 23), built from 10 variable MTRRs
dmesg_installer.txt:[    0.006843] ACPI: Reserving MCFG table memory at [mem 0x769a8fb0-0x769a8feb]
dmesg_installer.txt:[    0.007028] Faking a node at [mem 0x0000000000000000-0x000000087fbfffff]
dmesg_installer.txt:[    0.007032] NODE_DATA(0) allocated [mem 0x87fbd5000-0x87fbfffff]
dmesg_installer.txt:[    0.007178]   Normal   [mem 0x0000000100000000-0x000000087fbfffff]
dmesg_installer.txt:[    0.007183]   node   0: [mem 0x0000000100000000-0x000000087fbfffff]
dmesg_installer.txt:[    0.007186] Initmem setup node 0 [mem 0x0000000000001000-0x000000087fbfffff]
dmesg_installer.txt:[    0.058990] PM: hibernation: Registered nosave memory: [mem 0xfb000000-0xfb000fff]
dmesg_installer.txt:[    0.058990] PM: hibernation: Registered nosave memory: [mem 0xfb001000-0xfc7fffff]
dmesg_installer.txt:[    0.059002] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns
dmesg_installer.txt:[    0.156182] ... fixed-purpose events:   4
dmesg_installer.txt:[    0.007331] ... fixed-purpose events:   3
dmesg_installer.txt:[    0.271516] pci 0000:00:02.0: BAR 4 [io  0xefc0-0xefff]
dmesg_installer.txt:[    0.275394] pci 0000:00:1f.4: BAR 4 [io  0xefa0-0xefbf]
dmesg_installer.txt:[    0.396368] ACPI: bus type drm_connector registered
dmesg_installer.txt:[    0.434506] sysfb: VRAM smaller than advertised
dmesg_installer.txt:[    0.446779] Loaded X.509 cert 'Canonical Ltd. Secure Boot Signing (2021 v1): a8d54bbb3825cfb94fa13c9f8a594a195c107b8d'
dmesg_installer.txt:[    0.446812] Loaded X.509 cert 'Canonical Ltd. Secure Boot Signing (Ubuntu Core 2019): c1d57b8f6b743f23ee41f4f7ee292f06eecadfb9'
dmesg_installer.txt:[    8.499399] systemd[1]: Set up automount proc-sys-fs-binfmt_misc.automount - Arbitrary Executable File Formats File System Automount Point.
dmesg_installer.txt:[    8.518350] systemd[1]: Starting modprobe@drm.service - Load Kernel Module drm...
dmesg_installer.txt:[    8.519311] systemd[1]: Starting modprobe@efi_pstore.service - Load Kernel Module efi_pstore...
dmesg_installer.txt:[    8.545456] systemd[1]: modprobe@drm.service: Deactivated successfully.
dmesg_installer.txt:[    8.545520] systemd[1]: Finished modprobe@drm.service - Load Kernel Module drm.
dmesg_installer.txt:[    8.545646] systemd[1]: modprobe@efi_pstore.service: Deactivated successfully.
dmesg_installer.txt:[    8.545710] systemd[1]: Finished modprobe@efi_pstore.service - Load Kernel Module efi_pstore.
dmesg_installer.txt:[    9.261217] RAPL PMU: API unit is 2^-32 Joules, 4 fixed counters, 655360 ms ovfl timer
dmesg_installer.txt:[    9.692998] iwlwifi 0000:00:14.3: loaded firmware version 86.fb5c9aeb.0 so-a0-hr-b0-86.ucode op_mode iwlmvm
dmesg_installer.txt:[   10.255393] i915 0000:00:02.0: [drm] VT-d active for gfx access
dmesg_installer.txt:[   10.255554] i915 0000:00:02.0: [drm] Using Transparent Hugepages
dmesg_installer.txt:[   10.314944] i915 0000:00:02.0: [drm] Finished loading DMC firmware i915/adlp_dmc.bin (v2.20)
dmesg_installer.txt:[   10.369413] i915 0000:00:02.0: [drm] GT0: GuC firmware i915/adlp_guc_70.bin version 70.20.0
dmesg_installer.txt:[   10.369421] i915 0000:00:02.0: [drm] GT0: HuC firmware i915/tgl_huc.bin version 7.9.3
dmesg_installer.txt:[   10.387275] i915 0000:00:02.0: [drm] GT0: HuC: authenticated for all workloads
dmesg_installer.txt:[   10.388337] i915 0000:00:02.0: [drm] GT0: GUC: submission enabled
dmesg_installer.txt:[   10.388340] i915 0000:00:02.0: [drm] GT0: GUC: SLPC enabled
dmesg_installer.txt:[   10.388881] i915 0000:00:02.0: [drm] GT0: GUC: RC enabled
dmesg_installer.txt:[   10.389918] i915 0000:00:02.0: [drm] Protected Xe Path (PXP) protected content support initialized
dmesg_installer.txt:[   11.693190] fbcon: Taking over console
dmesg_installer.txt:[   12.042137] i915 0000:00:02.0: [drm] Skipping intel_backlight registration
dmesg_installer.txt:[   12.043661] [drm] Initialized i915 1.6.0 20230929 for 0000:00:02.0 on minor 0
dmesg_installer.txt:[   12.055115] fbcon: i915drmfb (fb0) is primary device
dmesg_installer.txt:[   12.090250] i915 0000:00:02.0: [drm] fb0: i915drmfb frame buffer device
dmesg_installer.txt:[   47.179665] audit: type=1326 audit(1712295526.344:80): auid=1000 uid=1000 gid=1000 ses=2 subj=snap.snapd-desktop-integration.snapd-desktop-integration pid=3503 comm="snapd-desktop-i" exe="/snap/snapd-desktop-integration/157/usr/bin/snapd-desktop-integration" sig=0 arch=c000003e syscall=203 compat=0 ip=0x71a29048a531 code=0x50000
dmesg_installer.txt:[   47.179673] audit: type=1326 audit(1712295526.344:81): auid=1000 uid=1000 gid=1000 ses=2 subj=snap.snapd-desktop-integration.snapd-desktop-integration pid=3503 comm="snapd-desktop-i" exe="/snap/snapd-desktop-integration/157/usr/bin/snapd-desktop-integration" sig=0 arch=c000003e syscall=141 compat=0 ip=0x71a29050978b code=0x50000
dmesg_installer.txt:[   47.206641] audit: type=1326 audit(1712295526.371:82): auid=1000 uid=1000 gid=1000 ses=2 subj=snap.snapd-desktop-integration.snapd-desktop-integration pid=3503 comm="snapd-desktop-i" exe="/snap/snapd-desktop-integration/157/usr/bin/snapd-desktop-integration" sig=0 arch=c000003e syscall=203 compat=0 ip=0x71a29048a531 code=0x50000
dmesg_installer.txt:[   47.206837] audit: type=1326 audit(1712295526.371:83): auid=1000 uid=1000 gid=1000 ses=2 subj=snap.snapd-desktop-integration.snapd-desktop-integration pid=3503 comm="snapd-desktop-i" exe="/snap/snapd-desktop-integration/157/usr/bin/snapd-desktop-integration" sig=0 arch=c000003e syscall=203 compat=0 ip=0x71a29048a531 code=0x50000
dmesg_installer.txt:[   47.206841] audit: type=1326 audit(1712295526.371:84): auid=1000 uid=1000 gid=1000 ses=2 subj=snap.snapd-desktop-integration.snapd-desktop-integration pid=3503 comm="snapd-desktop-i" exe="/snap/snapd-desktop-integration/157/usr/bin/snapd-desktop-integration" sig=0 arch=c000003e syscall=203 compat=0 ip=0x71a29048a531 code=0x50000
dmesg_installer.txt:[   47.206843] audit: type=1326 audit(1712295526.371:85): auid=1000 uid=1000 gid=1000 ses=2 subj=snap.snapd-desktop-integration.snapd-desktop-integration pid=3503 comm="snapd-desktop-i" exe="/snap/snapd-desktop-integration/157/usr/bin/snapd-desktop-integration" sig=0 arch=c000003e syscall=203 compat=0 ip=0x71a29048a531 code=0x50000
dmesg_installer.txt:[   47.206846] audit: type=1326 audit(1712295526.371:86): auid=1000 uid=1000 gid=1000 ses=2 subj=snap.snapd-desktop-integration.snapd-desktop-integration pid=3503 comm="snapd-desktop-i" exe="/snap/snapd-desktop-integration/157/usr/bin/snapd-desktop-integration" sig=0 arch=c000003e syscall=203 compat=0 ip=0x71a29048a531 code=0x50000
dmesg_installer.txt:[   47.206847] audit: type=1326 audit(1712295526.371:87): auid=1000 uid=1000 gid=1000 ses=2 subj=snap.snapd-desktop-integration.snapd-desktop-integration pid=3503 comm="snapd-desktop-i" exe="/snap/snapd-desktop-integration/157/usr/bin/snapd-desktop-integration" sig=0 arch=c000003e syscall=203 compat=0 ip=0x71a29048a531 code=0x50000
dmesg_installer.txt:[   47.206874] audit: type=1326 audit(1712295526.371:88): auid=1000 uid=1000 gid=1000 ses=2 subj=snap.snapd-desktop-integration.snapd-desktop-integration pid=3503 comm="snapd-desktop-i" exe="/snap/snapd-desktop-integration/157/usr/bin/snapd-desktop-integration" sig=0 arch=c000003e syscall=203 compat=0 ip=0x71a29048a531 code=0x50000
dmesg_installer.txt:[   47.206898] audit: type=1326 audit(1712295526.371:89): auid=1000 uid=1000 gid=1000 ses=2 subj=snap.snapd-desktop-integration.snapd-desktop-integration pid=3503 comm="snapd-desktop-i" exe="/snap/snapd-desktop-integration/157/usr/bin/snapd-desktop-integration" sig=0 arch=c000003e syscall=203 compat=0 ip=0x71a29048a531 code=0x50000
lsmod_installer.txt:xe                   2707456  0
lsmod_installer.txt:drm_gpuvm              45056  1 xe
lsmod_installer.txt:drm_exec               16384  2 drm_gpuvm,xe
lsmod_installer.txt:gpu_sched              61440  1 xe
lsmod_installer.txt:drm_suballoc_helper    16384  1 xe
lsmod_installer.txt:drm_ttm_helper         12288  1 xe
lsmod_installer.txt:drm_buddy              20480  2 xe,i915
lsmod_installer.txt:ttm                   110592  3 drm_ttm_helper,xe,i915
lsmod_installer.txt:drm_display_helper    253952  2 xe,i915
lsmod_installer.txt:cec                    98304  3 drm_display_helper,xe,i915
lsmod_installer.txt:i2c_algo_bit           16384  2 xe,i915
lsmod_installer.txt:efi_pstore             12288  0
lsmod_installer.txt:video                  73728  2 xe,i915
modules.builtin_initrd_finished_install.txt:kernel/fs/efivarfs/efivarfs.ko
modules.builtin_initrd_finished_install.txt:kernel/drivers/video/fbdev/core/fb.ko
modules.builtin_initrd_finished_install.txt:kernel/drivers/video/fbdev/core/cfbfillrect.ko
modules.builtin_initrd_finished_install.txt:kernel/drivers/video/fbdev/core/cfbcopyarea.ko
modules.builtin_initrd_finished_install.txt:kernel/drivers/video/fbdev/core/cfbimgblt.ko
modules.builtin_initrd_finished_install.txt:kernel/drivers/video/fbdev/core/fb_io_fops.ko
modules.builtin_initrd_finished_install.txt:kernel/drivers/video/fbdev/core/sysfillrect.ko
modules.builtin_initrd_finished_install.txt:kernel/drivers/video/fbdev/core/syscopyarea.ko
modules.builtin_initrd_finished_install.txt:kernel/drivers/video/fbdev/core/sysimgblt.ko
modules.builtin_initrd_finished_install.txt:kernel/drivers/video/fbdev/core/fb_sys_fops.ko
modules.builtin_initrd_finished_install.txt:kernel/drivers/video/fbdev/imsttfb.ko
modules.builtin_initrd_finished_install.txt:kernel/drivers/video/fbdev/asiliantfb.ko
modules.builtin_initrd_finished_install.txt:kernel/drivers/xen/xenbus/xenbus.ko
modules.builtin_initrd_finished_install.txt:kernel/drivers/xen/xenbus/xenbus_probe_frontend.ko
modules.builtin_initrd_finished_install.txt:kernel/drivers/xen/xen-acpi-processor.ko
modules.builtin_initrd_finished_install.txt:kernel/drivers/xen/grant-dma-ops.ko
modules.builtin_initrd_finished_install.txt:kernel/drivers/block/xen-blkfront.ko
modules.builtin_initrd_finished_install.txt:kernel/drivers/gpu/drm/drm.ko
modules.builtin_initrd_finished_install.txt:kernel/drivers/gpu/drm/drm_panel_orientation_quirks.ko
modules.builtin_initrd_finished_install.txt:kernel/drivers/gpu/drm/drm_shmem_helper.ko
modules.builtin_initrd_finished_install.txt:kernel/drivers/gpu/drm/drm_kms_helper.ko
modules.builtin_initrd_finished_install.txt:kernel/drivers/gpu/drm/drm_mipi_dsi.ko
modules.builtin_initrd_finished_install.txt:kernel/drivers/gpu/drm/tiny/simpledrm.ko
modules.builtin_initrd_finished_install.txt:kernel/drivers/net/phy/fixed_phy.ko
modules.builtin_initrd_finished_install.txt:kernel/drivers/net/xen-netfront.ko
modules.builtin_initrd_finished_install.txt:kernel/arch/x86/video/fbdev.ko

Analysis:

Result: on boot you see nothing after 12 seconds as per installer because nothing drives the framebuffer.

@daringer @alexgithublab : Was there an option to install proprietary drivers that was not ticked in in the installer?

tlaurion commented 5 months ago

Nothing much Heads can do here. Heads pass over the sysfb correctly (sysfb: VRAM smaller than advertised might be a coreboot bug) .

But that doesn't prevent the installer, with firmware + drm + i915+xe to drive the display fb.

Can you reinstall making sure firmware blobs are permitted and reply in this issue?

Only thing I can think of is the "primary display" config setting under coreboot, but highly doubt this would fix this which seems to be initamfs not being packed with required drivers

tlaurion commented 5 months ago

On nv41, Q4.2.1 dom0

$ lsmod  | grep -e fb -e drm -e efi
drm_buddy              20480  1 i915
drm_display_helper    237568  1 i915
cec                    90112  2 drm_display_helper,i915
$ sudo journalctl --boot 0  | grep kernel | grep -e fb -e drm -e efi
Apr 23 13:53:14 dom0 kernel: Xen: [mem 0x00000000fb000000-0x00000000fb000fff] reserved
Apr 23 13:53:14 dom0 kernel: clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns
Apr 23 13:53:14 dom0 kernel: clocksource: xen: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns
Apr 23 13:53:14 dom0 kernel: pci 0000:00:1f.4: reg 0x20: [io  0xefa0-0xefbf]
Apr 23 13:53:14 dom0 kernel: ACPI: bus type drm_connector registered
Apr 23 13:53:16 dom0 kernel: i915 0000:00:02.0: [drm] VT-d active for gfx access
Apr 23 13:53:16 dom0 kernel: i915 0000:00:02.0: [drm] Transparent Hugepage support is recommended for optimal performance on this platform!
Apr 23 13:53:16 dom0 kernel: i915 0000:00:02.0: [drm] Finished loading DMC firmware i915/adlp_dmc.bin (v2.20)
Apr 23 13:53:16 dom0 kernel: i915 0000:00:02.0: [drm] GT0: GuC firmware i915/adlp_guc_70.bin version 70.13.1
Apr 23 13:53:16 dom0 kernel: i915 0000:00:02.0: [drm] GT0: HuC firmware i915/tgl_huc.bin version 7.9.3
Apr 23 13:53:16 dom0 kernel: i915 0000:00:02.0: [drm] GT0: HuC: authenticated for all workloads
Apr 23 13:53:16 dom0 kernel: i915 0000:00:02.0: [drm] GT0: GUC: submission enabled
Apr 23 13:53:16 dom0 kernel: i915 0000:00:02.0: [drm] GT0: GUC: SLPC enabled
Apr 23 13:53:16 dom0 kernel: i915 0000:00:02.0: [drm] GT0: GUC: RC enabled
Apr 23 13:53:16 dom0 kernel: i915 0000:00:02.0: [drm] Protected Xe Path (PXP) protected content support initialized
Apr 23 13:53:18 dom0 kernel: i915 0000:00:02.0: [drm] Skipping intel_backlight registration
Apr 23 13:53:18 dom0 kernel: [drm] Initialized i915 1.6.0 20201103 for 0000:00:02.0 on minor 0
Apr 23 13:53:18 dom0 kernel: fbcon: i915drmfb (fb0) is primary device
Apr 23 13:53:18 dom0 kernel: fbcon: Deferring console take-over
Apr 23 13:53:18 dom0 kernel: i915 0000:00:02.0: [drm] fb0: i915drmfb frame buffer device

And QubesOS stays in the dark because no efifb driver can do the sysfb->efifb takeover because Xen in the way and not figured out. On non-Xen, efifb from sysfb should take the console right away. But efifb doesn't seem to kick in here prior of i915 being completely fired up.

alexgithublab commented 5 months ago

Still not booting with the proprietary drivers installed

tlaurion commented 5 months ago

lsmod_installer.txt dmesg_installer.txt

modules.builtin_initrd_finished_install.txt

So this might also be a ubuntu installer issue here?

@alexgithublab please provide same logs as @nestire here with blobs installed at installer phase.

Statement:

I'm trying to be as verbose possible so I don't have to fix, alone, everyone's issues from new platforms and all possible OSes, for all present and future sold platforms. Trying to leave traces for others to follow and be able to collaborate in code if not money.

More collaboration is needed for this project to thrive, unless workis expected to be paid from profits on said sold platforms.

"Free" as in free software should not be "free" as in free beer (from my bank account perspective).

I'm trying to give a fishing line here, not to be confounded with consenting unlimited supply of free fishes forever. Thanks for your understanding.

alexgithublab commented 5 months ago

here are the logs of ubuntu install with drivers installed :

dmesg-blob.txt lsmod-blob.txt find /lib/modules/$(uname -r) -type f -name '.ko' : modules-blob.txt

tlaurion commented 5 months ago

@alexgithublab : the gpu blobs are now present under installer's created initramfs? If so, why they are not taking over sysfb on boot is still unexplained. on my side.

https://github.com/linuxboot/heads/pull/1640#issuecomment-2079794121

find /lib/modules/$(uname -r) -type f -name '.ko' : modules-blob.txt

@alexgithublab, if i'm not mistaken, @nestire logs at

modules.builtin_initrd_finished_install.txt

were extracted from /boot's initramfs, where your log seems to come from the /lib from within the installer's shell. The former helps but unfortunately not the latter. Nothing tells us that those were packed under initramfs and made available when the system boots.

As asked under #1640 to @nestire : can we have a serial output here? the fact that we suffer from "black screen after kexec call" is problematic since we cannot get what the kernel is doing leading to not taking over the fb console. It happened in the past, sometimes plymouth/systemd takes for granted that for LUKS decryption key password, the console should be ready; you might have to type the LUKS password in the dark to get a working system to debug what is happening there.

@alexgithublab you also said you installed ubuntu without encryption before. If that still the case? If so, this means that my previous comment can be discarded and we need a way to dump kernel logs somewhere if we cannot get serial output.

tlaurion commented 5 months ago

@alexgithublab if this is testing latop, I would be interested to see if the roms currently built on CircleCI for alternative branch with coreboot+config+blobs versioned bump permits to boot your currently installed Ubuntu 24.04?

Those roms are built under https://app.circleci.com/pipelines/github/tlaurion/heads/2490/workflows/3add7434-cf6c-4935-a1c6-f88cdc3005d0 for https://github.com/tlaurion/heads/tree/nitrokey_board_unification_clean-enable_htop_validated_autoboot-novacustom_coreboot_version_bump branch

tlaurion commented 5 months ago

Cannot replicate failed build for ns50 locally with make make real.gitclean_keep_packages

Clearing CricleCI cache by setting CACHE_VERSION CircleCI environement variable to a unique (datestamp) value. Relaunching build.

nestire commented 4 months ago

Hey to summarize our testing: What works:

This works also on our 2.4.1 Release

What not Works:

I compared the initrd's of both installs and did not find any difference besides the added crypto modules

What I will test next is if this also accours with if you upgrade from ubuntu 22.04. -> 24.04 and will come back with that information

nestire commented 4 months ago

Ok did the Upgrade from 22.04 -> 23.04 -> 24.04 (directly to 24.04 is not working at the moment) the Kernel from 23.04 6.5.0-28 is working, the kernel 6.8.0-31 (24.04 hwe kernel) is not. So I have a working 24.04 install if choose the 6.5.0-28 kernel as default boot, tested this on our 2.4.1 install

tlaurion commented 4 months ago

Can you type the luks disk passphrase in the dark and then everything magically works @nestire on a fresh 24.04 install with luks+lvm is the question.

On Tue, Apr 30, 2024, 12:13 PM nestire @.***> wrote:

Ok did the Upgrade from 22.04 -> 23.04 -> 24.04 (directly to 24.04 is not working at the moment) the Kernel from 23.04 6.5.0-28 is working, the kernel 6.8.0-31 (24.04 hwe kernel) is not. So I have a working 24.04 install if choose the 6.5.0-28 kernel as default boot, tested this on our 2.4.1 install

— Reply to this email directly, view it on GitHub https://github.com/linuxboot/heads/issues/1641#issuecomment-2085822331, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAGKBMRLCQRM53F34K3VVALY767KNAVCNFSM6AAAAABGLL2UH6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAOBVHAZDEMZTGE . You are receiving this because you were mentioned.Message ID: @.***>

nestire commented 4 months ago

@tlaurion woaa that worked, with the upgrade install dmesg_2404_kernel_6.8.0-31.log

and also for a fresh install

tlaurion commented 4 months ago

@nestire now not sure how to tackle this but find what needs to be passed to board config through KERNEL_ADD additional statements to be passed at OS install so that equivalent of plymouth whatever loads kernel driver BEFORE the luks prompt....

Maybe you could share full journalctl --boot 0 on final working os fully booted?

Tldr: something fishy on Ubuntu side in the way they deal with FB readiness tests here, nothing Heads related unfortunately so prépare yourself to collaborate upstream (debian) or downstream (Ubuntu).

They might have started their legacy bios deprecation.

Expectation here would be efifb->simplefb->drm+fb->luks prompt, where at time of luks prompt, something is missing and why typing in the dark mitigated FB unreadyness.

Best you can do is let the prompt sit for a couple of seconds, type in the dark, and provide full logs here for upstream discussions on source of problem.

@nestire @alexgithublab you understand the problem?

alexgithublab commented 4 months ago

@tlaurion

dmseg.txt

Here are the logs after a successful boot typing the Luks pwd during the black screen.

Note that after typing the Luks pwd I have a little bit of logs showing on the screen which are "mei_me..." so the relevant logs must be around [25.000000]

nestire commented 4 months ago

Here is the journal jounrnalctl_boot_0.log

What I see is the:sysfb: VRAM smaller than advertised

tlaurion commented 4 months ago

Here is the journal jounrnalctl_boot_0.log

What I see is the:sysfb: VRAM smaller than advertised

Yeah, that's a dead end. I see this in all boards being libgfxinit based as well as gop based FB init. So not thought to be source of the problem.

As stated before, the fact that you can type the luks passphrase in the dark and then it works shows that it's not heads related issue but something missing in kernel_add board config or a bug upsream. What, I do not know.

tlaurion commented 2 months ago

Here is the journal jounrnalctl_boot_0.log

What I see is the:sysfb: VRAM smaller than advertised

@nestire as from 24.02.01 coreboot bump results you posted at https://github.com/linuxboot/heads/pull/1723#issuecomment-2247615280, and Nitrokey providing Ubuntu OEM images, this needs revisiting to support Nitrokey users.

sysfb (coreboot) might be causing issues, not sure why on Ubuntu but not other OSes. This is coreboot <-> Ubuntu issues needing deeper troubleshooting where Heads is just using sysfb+efifb from GOP/libgfxinit so we need to bring both support communities together to resolve this issue.

Willing to open bug reports upstream and cross-link them here for tracking so that Nitrokey user base has professional UX?

nestire commented 2 months ago

@tlaurion yeah I'm up to put some more effort into this, but I'm also at a dead end after unsuccessfully testing some kernel options, and kernels (mainline also did not work) etc.. Also I don't find other people online who have the same problem so this seems limited to heads/coreboot. There is one issue that seems to be a similar problem in fedora 38 but I'm not sure if it is actually the same https://discussion.fedoraproject.org/t/luks-encryption-password-screen-not-showing/85683 . And I would need to test this which I did not yet find the time to do.

I'm not convinced that we get canonical to act on this, but with coreboot this might be a different story. We will discuss this, and cross link the ticket for coreboot here

daringer commented 2 months ago

naive question: did anybody crosscheck if other similarly "recent" kernels show the same behavior ?

tlaurion commented 2 months ago

naive question: did anybody crosscheck if other similarly "recent" kernels show the same behavior ?

Heads or kexec'ed final OS kernel? As far as I know, behavior is not present under Debian nor Fedora and last investigation from provided logs showed that the console linked to framebuffer under final Ubuntu initrd was missing something (LUKS decryption prompt is under initrd) and where FB completely owned happens after the prompt.

Recap without rereading logs:

So something is not kosher on Ubuntu side of things in their legacy bios support with efifb->drm+3d FB takeover. At least, that's the current hypothesis.

Maybe, I say maybe, kexec and/or coreboot fixed FB landover more nicely and/or Ubuntu Legacy BIOS deprecation plan went further ahead of other OSes, but intuition is something missing under initrd to play nicely since not affecting everything and typing LUKS in the dark, or setting DUK in Heads mitigate the whole issue.

nestire commented 1 month ago

Ok based on this I had a look at the Kernel Config in Ubuntu and saw that CONFIG_FB_EFI was not set set this to CONFIG_FB_EFI=y and recompiled the kernel see here: https://git.launchpad.net/~ubuntu-kernel/ubuntu/+source/linux/+git/noble/tree/debian.master/config/annotations?id=8b8990d22605ad0d2c20e00de9d2ed1294cffa62#n237

solved it in part (plymouth did not start but that might have to do with how i build the kernel) so will try with this information to open a ticket to see if they are willing to change this back

thx @tlaurion for the summary that helped a lot

tlaurion commented 1 month ago

so will try with this information to open a ticket to see if they are willing to change this back

Please cross link opened issue to this one and this one to upstream opened issue there and ping me where needed.

nestire commented 1 month ago

ok the changes to this where discussed here:

https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1965303

commented there instead of opening a new bug

tlaurion commented 1 month ago

ok the changes to this where discussed here:

https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1965303

commented there instead of opening a new bug

@daringer @nestire @jans23 From referred launchpad

Thanks, but this bug is closed. Please open a new Launchpad bug.

tlaurion commented 1 month ago

@nestire ping

tlaurion commented 1 month ago

@daringer @nestire @jans23 : ping

Discussed at https://matrix.to/#/!RNcjJXCGHiyxXCHpKv:matrix.org/$YSYqj1K7qdJvbjT4NRvEYn4t3-9tl9vvyJdEoWHjOgs?via=matrix.org&via=nitro.chat&via=matrix.3mdeb.com (this is Dasharo Premier Support channel: accessible only to subscribers)

End user asked what OSes were supported by Heads. TLDR: any linux OS respecting legacy bios deprecation plan, and providing efifb+simplefb/simpledrm in initramfs per https://fedoraproject.org/wiki/Changes/DeprecateLegacyBIOS

What needs to be added under https://osresearch.net/InstallingOS/#generic-os-installation ?

@nestire : bug to be opened upstream referring to https://github.com/linuxboot/heads/issues/1641#issuecomment-2252658288

  • basic fb drivers/systemd/scripts are supposed to collaborate nicely to provide early console output (this is where Ubuntu fails here not providing efifb through CONFIG_FB_EFI=y or packing module under intramfs with CONFIG_FB_EFI=m).