Talos II - 0.6.1 - Workstation - No USB keyboard support and no terminal redirection (expected withFBwhiptail)

[tlaurion]

Dasharo version 0.6

Dasharo variant

Workstation

Affected component(s) or functionality

Brief summary USB keyboard doesn't work and there is no fbwhiptail redirection to bmc terminal (normal).

User has to use external laptop to control remotely over BMC yo control and see what is on connected over VGA port until final OS boots.

How reproducible 100%

How to reproduce

Boot workstation, type on usb keyboard to realise that nothing can be typed on keyboard to Heads.

Expected behavior Have USB keyboard working to control workstation from boot.

Actual behavior Input can only be done through bmc emulated usb to control workstation.

Screenshots

Additional context USB_KEYBOARD (HID driver) should probably be defined under board config and compiled as module inside of linux configuration.

Solutions you've tried Using server variant to test 0.6.1 as of now

[tlaurion]

@SergiiDmytruk : The following should fix the current issue https://github.com/osresearch/heads/compare/master...tlaurion:heads:talos2_cryptsetup2_usb_keyboard_for_workstation#diff-621bfa30b71e6d738181d2efc0cb94da0922a89dfc6831656518b44a17d62a1aR40-R42

Which enables usb as early as possible in init: https://github.com/osresearch/heads/blob/master/initrd/init#L82-L85 And which loads usb_hid (insmod wrapper) when enabling usb: https://github.com/osresearch/heads/blob/master/initrd/etc/functions#L126-L131

And where insmod wrapper measures and extends PCR prior of actually loading the module: https://github.com/osresearch/heads/blob/master/initrd/sbin/insmod

I did not test USB keyboard on Talos II Workstation with those changes yet. Please reuse proposed changes on first link as needed.

[SergiiDmytruk]

The following should fix the current issue osresearch/heads@master...tlaurion:heads:talos2_cryptsetup2_usb_keyboard_for_workstation#diff-621bfa30b71e6d738181d2efc0cb94da0922a89dfc6831656518b44a17d62a1aR40-R42

I thought there was some other option and only now noticed that it's the same CONFIG_USB_KEYBOARD=y (lines 37 and 41 now set it twice), hence no change in behaviour.

[tlaurion]

No, as said over matrix off-channel discussion, even recompiling with usbhid in-kernel doesn't change the behavior for workstation.

This is why it was said that behavior seems to be a kernel bad behavior.

What is weird is that the server boad configuration receives input with usbhid compiled as module, as expected, where workstation doesn't. I thought workstation didn't load usbhid module and was wrong. It does. But still doesn't interact with USB keyboard on my side. It does on yours?

So the insight is that its not the kernel module nor heads added logic, but something along framebuffer (the difference between server and workstation boards) where kernel options which made kgpe-d16 server and workstation different cannot be passed in coreboot config and defined in skiboot. Kgpe-d16 workstation board with USB keyboard works.

Any update on testing corresponding options from skiboot? If that fix the behavior, then coreboot needs to be able to accept those so that a different coreboot config can be made for server and workstation, so that a dual console can be configured.

Please take a look at kgpe-d16 coreboot configuration and passed kernel options. The idea here should be to have two console defined, the latest being the main one. This is important for Linux.

[SergiiDmytruk]

I also see:

[  145.027621] mm/pgtable-generic.c:50: bad pmd c00600c2806001be.

And the behaviour seems similar to yours: in BMC KVM keyboard doesn't work until recovery is selected via serial console. However, there is no usbhid error for the hardware keyboard:

[   90.423558] input: Logitech USB Keyboard as /devices/pci0003:00/0003:00:00.0/0003:01:00.0/usb1/1-2/1-2:1.0/0003:046D:C31C.0001/input/input0
[   90.491857] hid-generic 0003:046D:C31C.0001: input: USB HID v1.10 Keyboard [Logitech USB Keyboard] on usb-0003:01:00.0-2/input0
[   90.515588] input: Logitech USB Keyboard Consumer Control as /devices/pci0003:00/0003:00:00.0/0003:01:00.0/usb1/1-2/1-2:1.1/0003:046D:C31C.0002/input/input1
[   90.581883] input: Logitech USB Keyboard System Control as /devices/pci0003:00/0003:00:00.0/0003:01:00.0/usb1/1-2/1-2:1.1/0003:046D:C31C.0002/input/input2

Any update on testing corresponding options from skiboot? If that fix the behavior, then coreboot needs to be able to accept those so that a different coreboot config can be made for server and workstation, so that a dual console can be configured.

Any option specified there can be specified in config/linux-talos-2_workstation.config, which currently says:

CONFIG_CMDLINE="console=tty0 console=hvc0 powersave=off quiet"

I did try making coreboot's CONFIG_LINUX_COMMAND_LINE work, but something in Kconfig prevents it from appearing in .config.

[SergiiDmytruk]

With console=hvc0 console=tty0 nothing gets displayed anywhere. console=tty0 console=hvc0 console=fb0 works as console=tty0 console=hvc0.

Linux v5.10.5 shows no difference compared to v5.5. Linux v6.0.1 behaves the same in my tests, but bad pmd error is gone. Maybe give this build a try (zImage.bundled-v6.0.1.gz) to see if it behaves differently.

[tlaurion]

Last console specified has impact for input https://tldp.org/HOWTO/Remote-Serial-Console-HOWTO/configure-kernel.html (hint from @miczyg1 from community call)

Will re-read, had to deal with that with kgpe-d16/qemu boards)

[miczyg1]

@SergiiDmytruk

I did try making coreboot's CONFIG_LINUX_COMMAND_LINE work, but something in Kconfig prevents it from appearing in .config

What might be happening here is that you try to set this manually while none of Linux payload nor LinuxBoot payload is selected first in the configuration. What we have is CONFIG_PAYLOAD_SKIBOOT=y so it definitely won't work.

[miczyg1]

Are we sure tty0 is created on POWER systems? Kernel documentation says hvc0 is for POWER machines: https://github.com/torvalds/linux/blob/master/Documentation/admin-guide/kernel-parameters.txt#L738

[tlaurion]

dmesg console output from Heads

~ # dmesg | grep console
[    0.000000] printk: bootconsole [udbg0] enabled
[    0.000000] Kernel command line: console=tty0 console=hvc0 powersave=off quiet
[    0.000082] printk: console [tty0] enabled
[    0.000084] printk: console [hvc0] enabled
[    0.000086] printk: bootconsole [udbg0] disabled
[    4.413394] hvc0: raw protocol on /ibm,opal/consoles/serial@0 (boot console)

[SergiiDmytruk]

What might be happening here is that you try to set this manually while none of Linux payload nor LinuxBoot payload is selected first in the configuration. What we have is CONFIG_PAYLOAD_SKIBOOT=y so it definitely won't work.

I've also added

config LINUX_COMMAND_LINE
    string

to src/mainboard/raptor-cs/talos-2/Kconfig and thought it should work like this, but something is missing or reusing the same name is wrong.

[miczyg1]

Resuing it will not work IIRC.

[SergiiDmytruk]

Saw that it's already defined by two payloads and definitions are identical, copy&pasting that option made it work. I guess two lines above just don't constitute a proper definition. Image with CONFIG_LINUX_COMMAND_LINE= working: coreboot.rom.signed.ecc.gz.

[tlaurion]

@SergiiDmytruk I looked into https://app.circleci.com/pipelines/github/SergiiDmytruk/heads and see a commit from 3 days ago for flashrom branch.

Could you link here a branch/commit/CircleCI build so I know what i'm testing? Ideally to speed up/entice testing process, it would be nice if those were including a working branch link, so I can test a bunch of fixes all at once, lowering the time needed to test and also easying builds testing from the community we are trying to have test the builds and report maybe other issues i'm seldom reporting for the moment.

---

As of now, I understand that

• this coreboot passes the proper CONFIG_LINUX_COMMAND_LINE https://github.com/Dasharo/dasharo-issues/files/9786328/coreboot.rom.signed.ecc.gz (this post) . I understand that this coreboot should pass kernel config arguments to kernel, and the zimage to not have the kernel boot arguments inside of its .config file.
• This zimage is supposed to be a kernel Linux v6.0.1 which removes bad pmd errors for workstation board, but does not support usb keyboard properly yet: https://github.com/Dasharo/dasharo-issues/files/9768833/zImage.bundled-v6.0.1.gz (this post). I also understand that the command line is in the kernel config, not passed from coreboot.

You want me to test those separately/conjointly (doesn't seem to make sense).

[SergiiDmytruk]

You want me to test those separately/conjointly?

Can be tested at the same time, one shouldn't affect the other.

coreboot image is the latest release + patch to make CONFIG_LINUX_COMMAND_LINE work.

I looked into https://app.circleci.com/pipelines/github/SergiiDmytruk/heads and see a commit from 3 days ago for flashrom branch.

It's a build of https://github.com/osresearch/heads/pull/1222

zImage.bundled-v6.0.1.gz is actually built on that flashrom branch as well. This and coreboot built locally.

This zimage is supposed to be a kernel Linux v6.0.1 which removes bad pmd errors for workstation board, but does not support usb keyboard properly yet

The behaviour might be different in some way, I'm not sure. If keyboard still doesn't work, then bad pmd errors are not the cause.

[SergiiDmytruk]

I should add that coreboot image was built with CONFIG_LINUX_COMMAND_LINE="console=tty0 console=hvc0 quiet". And not sure why I provided it if the point of the patch is to make compile-time options work... Runtime behaviour won't really change.

[tlaurion]

Ok.

So taking artifacts from https://app.circleci.com/pipelines/github/SergiiDmytruk/heads/88/workflows/6b2ef291-f306-4e3f-a6d7-29986a3c54e6/jobs/1669

• https://output.circle-artifacts.com/output/job/075c9abd-a139-4af5-aa2c-b4e31a236619/artifacts/0/build/ppc64/talos-2_workstation/heads-talos-2_workstation-v0.2.0-1267-g73bc621.bootblock
• https://output.circle-artifacts.com/output/job/075c9abd-a139-4af5-aa2c-b4e31a236619/artifacts/0/build/ppc64/talos-2_workstation/heads-talos-2_workstation-v0.2.0-1267-g73bc621.rom
• https://output.circle-artifacts.com/output/job/075c9abd-a139-4af5-aa2c-b4e31a236619/artifacts/0/build/ppc64/talos-2_workstation/zImage.bundled

Will report under https://github.com/osresearch/heads/pull/1222 first.

[tlaurion]

I should add that coreboot image was built with CONFIG_LINUX_COMMAND_LINE="console=tty0 console=hvc0 quiet". And not sure why I provided it if the point of the patch is to make compile-time options work... Runtime behaviour won't really change.

Not sure I follow. So coreboot passes the kernel boot arguments in your coreboot image above (this ticket). This overrides the zImage.bundle arguments? so coreboot image should be tested to see if dmesg removes powersave (to validate).

[SergiiDmytruk]

So coreboot passes the kernel boot arguments in your coreboot image above (this ticket). This overrides the zImage.bundle arguments?

If CONFIG_LINUX_COMMAND_LINE is defined in coreboot config of Heads, the code will add it to the device tree at /chosen/bootargs and it will override CONFIG_CMDLINE in Linux config of Heads.

so coreboot image should be tested to see if dmesg removes powersave (to validate).

It seems to work (cat /proc/cmdline is as expected, same for /proc/device-tree/chosen/bootargs). As I said, there isn't much to test in it. The patch just makes CONFIG_LINUX_COMMAND_LINE work in config/coreboot-talos-2.config, which is what you wanted for playing with kernel parameters. Here's the patch if you want to use it:

 src/mainboard/raptor-cs/talos-2/Kconfig |  6 ++++++
 src/soc/ibm/power9/chip.c               | 11 +++++++++++
 2 files changed, 17 insertions(+)

diff --git a/src/mainboard/raptor-cs/talos-2/Kconfig b/src/mainboard/raptor-cs/talos-2/Kconfig
index ca662f2eab..d2d8273226 100644
--- a/src/mainboard/raptor-cs/talos-2/Kconfig
+++ b/src/mainboard/raptor-cs/talos-2/Kconfig
@@ -89,4 +89,10 @@ config DRAM_SIZE_MB
    int
    default 32768

+config LINUX_COMMAND_LINE
+   string "Linux command line"
+   default ""
+   help
+     A command line to add to the Linux kernel.
+
 endif #  BOARD_RAPTOR_CS_TALOS_2
diff --git a/src/soc/ibm/power9/chip.c b/src/soc/ibm/power9/chip.c
index 88c08af839..f4769b45fb 100644
--- a/src/soc/ibm/power9/chip.c
+++ b/src/soc/ibm/power9/chip.c
@@ -343,6 +343,16 @@ static void add_reserved_memory_node(struct device_tree *tree, uint64_t start, u
    dt_add_reg_prop(node, &start, &size, 1, 2, 2);
 }

+static void add_cmdline(struct device_tree *tree)
+{
+#ifdef CONFIG_LINUX_COMMAND_LINE
+   struct device_tree_node *node;
+
+   node = dt_find_node_by_path(tree, "/chosen", NULL, NULL, 1);
+   dt_add_string_prop(node, "bootargs", (const char *)CONFIG_LINUX_COMMAND_LINE);
+#endif
+}
+
 static void add_memory_nodes(struct device_tree *tree)
 {
    struct mem_map map;
@@ -576,6 +586,7 @@ static int dt_platform_update(struct device_tree *tree, uint8_t chips)

    validate_dt(tree, chips);

+   add_cmdline(tree);
    add_memory_nodes(tree);
    add_dimm_sensor_nodes(tree, chips);
    add_cb_fdt_data(tree);

Will need to merge it to have it in the next release.

[tlaurion]

Ok, so I understand that zImage.bundled is the next step here on top of https://github.com/osresearch/heads/pull/1222

Doing:

root@talos:/tmp/images/issue193# pflash  -F ../../talos.pnor -f -P BOOTKERNEL -p zImage.bundled-v6.0.1 && mboxctl --backend file:/tmp/talos.pnor
About to program "zImage.bundled-v6.0.1" at 0x022a1000..0x02f39068 !
Programming & Verifying...
[==================================================] 100%
Updating actual size in partition header...
SetBackend: Success
root@talos:/tmp/images/issue193# obmcutil poweroff && obmcutil poweron && obmc-console-client
st type 0x0.

coreboot--TIMELESS--LESSTIME--Heads-v0.2.0-1267-g73bc621 Thu Jan  1 00:00:00 UTC 1970 bootblock starting (log level: 7)...
FFS header at 0x80060300ffff7000
PNOR base at 0x80060300fc000000
HBI partition has ECC
HBI is in 0x00426200 through 0x0175f037
FMAP: Found "FLASH" version 1.1 at 0x20000.
FMAP: base = 0x0 size = 0x200000 #areas = 4
FMAP: area COREBOOT found @ 20200 (1965568 bytes)
CBFS: mcache @0xf8231000 built for 10 files, used 0x1f0 of 0x2000 bytes
CBFS: Found 'fallback/romstage' @0x80 size 0x124a1 in mcache @0xf823102c
BS: bootblock times (exec / console): total (unknown) / 2 ms

coreboot--TIMELESS--LESSTIME--Heads-v0.2.0-1267-g73bc621 Thu Jan  1 00:00:00 UTC 1970 romstage starting (log level: 7)...
IPMI: romstage PNP BT 0xe4
Get BMC self test result...Function Not Implemented
Initializing IPMI BMC watchdog timer
IPMI BMC watchdog initialized and started.
Initializing FSI...
Initialized FSI (chips mask: 0x01)
Building MVPDs...
starting istep 8.1
starting istep 8.2
starting istep 8.3
starting istep 8.4
starting istep 8.9
Base epsilon values read from table:
 R_T[0] = 22
 R_T[1] = 22
 R_T[2] = 103
 W_T[0] = 5
 W_T[1] = 33
Scaled epsilon values based on +20 percent guardband:
 R_T[0] = 27
 R_T[1] = 27
 R_T[2] = 124
 W_T[0] = 6
 W_T[1] = 40
starting istep 8.10
starting istep 8.11
starting istep 9.2
starting istep 9.4
starting istep 9.6
starting istep 9.7
starting istep 10.1
starting istep 10.6
starting istep 10.10
starting istep 10.12
starting istep 10.13
FFS header at 0x80060300ffff7000
PNOR base at 0x80060300fc000000
MEMD partition has ECC
MEMD is in 0x03cef200 through 0x03cfb917
I2C transfer failed to complete (0x04011f0104000000)
No memory DIMM at address 51
I2C transfer failed to complete (0x04011f0104000000)
No memory DIMM at address 52
I2C transfer failed to complete (0x04011f0104000000)
No memory DIMM at address 53
I2C transfer failed to complete (0x04011f0104000000)
No memory DIMM at address D4
I2C transfer failed to complete (0x04011f0104000000)
No memory DIMM at address D5
I2C transfer failed to complete (0x04011f0104000000)
No memory DIMM at address D6
I2C transfer failed to complete (0x04011f0104000000)
No memory DIMM at address D7
SPD @ 0x50
SPD: module type is DDR4
SPD: module part number is M393A1K43BB0-CRC    
SPD: banks 16, ranks 1, rows 16, columns 10, density 8192 Mb
SPD: device width 8 bits, bus width 64 bits
SPD: module size is 8192 MB (per channel)
starting istep 13.1
starting istep 13.2
starting istep 13.3
starting istep 13.4
starting istep 13.5
starting istep 13.6
starting istep 13.7
starting istep 13.8
starting istep 13.9
starting istep 13.10
CCS took 2 us (3 us timeout), 1 instruction(s)
CCS took 2 us (2 us timeout), 14 instruction(s)
RCD dump for I2C address 0x58:
0xf820fa30: 80 b3 40 42 30 00 00 00 02 01 00 03 cb e3 c0 0d  ..@B0...........
0xf820fa40: 00 00 39 00 00 00 00 00 00 00 07 00 00 00 00 00  ..9.............
starting istep 13.11
CCS took 2 us (7 us timeout), 2 instruction(s)
Write Leveling starting
CCS took 12 us (92 us timeout), 5 instruction(s)
Write Leveling done
Initial Pattern Write starting
CCS took 5 us (38 us timeout), 5 instruction(s)
Initial Pattern Write done
DQS alignment starting
CCS took 11 us (44 us timeout), 1 instruction(s)
DQS alignment done
Read Clock Alignment starting
CCS took 8 us (82 us timeout), 1 instruction(s)
Read Clock Alignment done
Read Centering starting
CCS took 37 us (120 us timeout), 1 instruction(s)
Read Centering done
Write Centering starting
CCS took 9118 us (11314 us timeout), 7 instruction(s)
Write Centering done
Coarse write/read starting
CCS took 5 us (24 us timeout), 1 instruction(s)
Coarse write/read done
starting istep 13.12
starting istep 13.13
starting istep 14.1
starting istep 14.2
starting istep 14.3
Initializing PEC0...
Initializing PEC1...
Initializing PEC2...
Initializing PHB0...
Initializing PHB1...
Initializing PHB2...
Initializing PHB3...
Initializing PHB4...
Initializing PHB5...
starting istep 14.4
starting istep 14.5
0xF000F = 221d104900008040
CBMEM:
IMD: root @ 0xffeff000 254 entries.
IMD: root @ 0xffefec00 62 entries.
FMAP: area COREBOOT found @ 20200 (1965568 bytes)
FFS header at 0x80060300ffff7000
PNOR base at 0x80060300fc000000
HBI partition has ECC
HBI is in 0x00426200 through 0x0175f037
CBFS: Found 'fallback/ramstage' @0x12580 size 0xcca9 in mcache @0xf8231080
BS: romstage times (exec / console): total (unknown) / 17 ms

coreboot--TIMELESS--LESSTIME--Heads-v0.2.0-1267-g73bc621 Thu Jan  1 00:00:00 UTC 1970 ramstage starting (log level: 7)...
Enumerating buses...
Root Device scanning...
DD21, boot core: 18
FFS header at 0x80060300ffff7000
PNOR base at 0x80060300fc000000
HCODE partition has ECC
HCODE is in 0x01a82200 through 0x01b821ff
FFS header at 0x80060300ffff7000
PNOR base at 0x80060300fc000000
OCC partition has ECC
OCC is in 0x03822200 through 0x039221ff
Base epsilon values read from table:
 R_T[0] = 22
 R_T[1] = 22
 R_T[2] = 103
 W_T[0] = 5
 W_T[1] = 33
Scaled epsilon values based on +20 percent guardband:
 R_T[0] = 27
 R_T[1] = 27
 R_T[2] = 124
 W_T[0] = 6
 W_T[1] = 40
FFS header at 0x80060300ffff7000
PNOR base at 0x80060300fc000000
WOFDATA partition has ECC
WOFDATA is in 0x039ea200 through 0x03c94ca7
Matching WOF tables section not found, disabling WOF
Safe mode freq = 1833412 kHZ, voltage = 711 mv
starting istep 15.2
starting istep 15.3
starting istep 15.4
Starting PM complex...
Attempting PGPE activation...
PGPE was activated successfully
Done starting PM complex
Activating OCC...
Done activating OCC
starting istep 16.1
XIVE configured, entering dead man loop
ASSERTION ERROR: file '(filenames not available on timeless builds)', line 404
starting istep 18.11
starting istep 18.12
CPU_CLUSTER: 0 enabled
PNP: 00e4.0 enabled
scan_bus: bus Root Device finished in 3072 msecs
done
BS: BS_DEV_ENUMERATE run times (exec / console): 3066 / 7 ms
Allocating resources...
Reading resources...
CPU_CLUSTER: 0 missing read_resources
Done reading resources.
CPU_CLUSTER: 0 missing set_resources
PNP: 00e4.0 00 <- [0x00000000e4 - 0x00000000e6] size 0x00000003 gran 0x00 io
Done setting resources.
Done allocating resources.
BS: BS_DEV_RESOURCES run times (exec / console): 0 / 1 ms
Enabling resources...
done.
Initializing devices...
PNP: 00e4.0 init
IPMI: PNP BT 0xe4
Get BMC self test result...Function Not Implemented
IPMI: Found man_id 0x6cb0000, prod_id 0x1000000
IPMI: Version 2.0
PNP: 00e4.0 init finished in 19 msecs
Devices initialized
BS: BS_DEV_INIT run times (exec / console): 19 / 1 ms
Finalize devices...
Devices finalized
Writing coreboot table at 0xffed9000
 0. 0000000000000000-00000000f8207fff: RAM
 1. 00000000f8208000-00000000f820ffff: RAMSTAGE
 2. 00000000f8210000-00000000f8ffffff: RAM
 3. 00000000f9000000-00000000f929bfff: RAMSTAGE
 4. 00000000f929c000-00000000ffed8fff: RAM
 5. 00000000ffed9000-00000000ffefffff: CONFIGURATION TABLES
 6. 00000000fff00000-00000001ff3fffff: RAM
 7. 00000001ff400000-00000001ffffffff: RESERVED
FMAP: area COREBOOT found @ 20200 (1965568 bytes)
FFS header at 0x80060300ffff7000
PNOR base at 0x80060300fc000000
HBI partition has ECC
HBI is in 0x00426200 through 0x0175f037
Wrote coreboot table at: 0xffed9000, 0x214 bytes, checksum 3d45
coreboot table: 556 bytes.
IMD ROOT    0. 0xffeff000 0x00001000
IMD SMALL   1. 0xffefe000 0x00001000
CONSOLE     2. 0xffede000 0x00020000
MEM INFO    3. 0xffedb000 0x000022b0
COREBOOT    4. 0xffed9000 0x00002000
IMD small region:
  IMD ROOT    0. 0xffefec00 0x00000400
  RO MCACHE   1. 0xffefea00 0x000001f0
  FMAP        2. 0xffefe920 0x000000e0
BS: BS_WRITE_TABLES run times (exec / console): 0 / 6 ms
CBFS: Found 'fallback/payload' @0x257c0 size 0x632f9 in mcache @0xffefeb84
Checking segment from ROM address 0xf8380000
Checking segment from ROM address 0xf838001c
Checking segment from ROM address 0xf8380038
Loading segment from ROM address 0xf8380000
  code (compression=1)
  New segment dstaddr 0x00000000 memsize 0x173c00 srcaddr 0xf8380054 filesize 0x632a5
Loading Segment: addr: 0x00000000 memsz: 0x0000000000173c00 filesz: 0x00000000000632a5
using LZMA
Loading segment from ROM address 0xf838001c
  BSS 0x00300000 (113776 byte)
Loading Segment: addr: 0x00300000 memsz: 0x000000000001bc70 filesz: 0x0000000000000000
it's not compressed!
Clearing Segment: addr: 0x0000000000300000 memsz: 0x000000000001bc70
Loading segment from ROM address 0xf8380038
  Entry Point 0x00002570
BS: BS_PAYLOAD_LOAD run times (exec / console): 297 / 4 ms
Jumping to boot code at 0x00002570(0xffed9000)
CBFS: Found '1-cpu.dtb' @0x1f700 size 0x24e7 in mcache @[   11.295061279,5] OPAL v6.3-rc2-32-gfa060c2c starting...
[   11.295064862,7] initial console log level: memory 7, driver 5
[   11.295066911,6] CPU: P9 generation processor (max 4 threads/core)
[   11.295068693,7] CPU: Boot CPU PIR is 0x0048 PVR is 0x004e1201
[   11.295071197,7] OPAL table: 0x30108230 .. 0x30108810, branch table: 0x30002000
[   11.295074208,7] Assigning physical memory map table for nimbus
[   11.295076587,7] FDT: Parsing fdt @0xf90a55f8
[   11.295775785,6] CHIP: Initialised chip 0 from xscom@603fc00000000
[   11.295800236,6] P9 DD2.10 detected
[   11.295801805,5] CHIP: Chip ID 0000 type: P9N DD2.10
[   11.295803867,7] XSCOM: Base address: 0x603fc00000000
[   11.295812587,7] XSTOP: XSCOM addr = 0x5012000, FIR bit = 31
[   11.295814712,6] MFSI 0:0: Initialized
[   11.295816123,6] MFSI 0:2: Initialized
[   11.295817520,6] MFSI 0:1: Initialized
[   11.295871312,6] LPC: LPC[000]: Initialized
[   11.295872741,7] LPC: access via MMIO @0x6030000000000
[   11.295885623,7] LPC: Default bus on chip 0x0
[   11.295892421,7] CPU: New max PIR set to 0x57
[   11.295922847,6] MEM: parsing reserved memory from node /reserved-memory
[   11.295927046,7] HOMER: Init chip 0
[   11.295929286,7]   PBA BAR0 : 0x00000001ff400000
[   11.295931049,7]   PBA MASK0: 0x0000000000300000
[   11.295932712,7]   HOMER Image at 0x1ff400000 size 4MB
[   11.295935524,7]   PBA BAR2 : 0x00000001ff800000
[   11.295937152,7]   PBA MASK2: 0x0000000000700000
[   11.295938791,7]   OCC Common Area at 0x1ff800000 size 8MB
[   11.295941137,7] CPU: decrementer bits 56
[   11.295945650,6] CPU: CPU from DT PIR=0x0048 Server#=0x48 State=3
[   11.295951099,6] CPU:  4 secondary threads
[   11.295954120,6] CPU: CPU from DT PIR=0x004c Server#=0x4c State=3
[   11.295959662,6] CPU:  4 secondary threads
[   11.295962680,6] CPU: CPU from DT PIR=0x0050 Server#=0x50 State=3
[   11.295968167,6] CPU:  4 secondary threads
[   11.295971153,6] CPU: CPU from DT PIR=0x0054 Server#=0x54 State=3
[   11.295976557,6] CPU:  4 secondary threads
[   11.296024248,6] PLAT: AST SIO unavailable!
[   11.296036616,7] UART: Using LPC IRQ 4
[   11.298110430,5] PLAT: Detected Talos platform
[   11.298151720,5] PLAT: Detected BMC platform ast2500:openbmc
[   11.313120684,5] CPU: All 16 processors called in...
[   11.406021856,7] LPC: Routing irq 10, policy: 0 (r=1)
[   11.406022947,7] LPC: SerIRQ 10 using route 0 targetted at OPAL
[   12.308909331,5] HIOMAP: Negotiated hiomap protocol v2
[   12.308966785,5] HIOMAP: Block size is 4KiB
[   12.308998193,5] HIOMAP: BMC suggested flash timeout of 8s
[   12.309035218,5] HIOMAP: Flash size is 64MiB
[   12.309072521,5] HIOMAP: Erase granule size is 4KiB
[   14.216440172,5] FLASH: Found system flash: (unnamed) id:0
[   15.007138364,5] STB: secure boot not supported
[   15.007196640,5] STB: trusted boot not supported
[   15.007232458,7] LPC: Routing irq 4, policy: 0 (r=1)
[   15.007233389,7] LPC: SerIRQ 4 using route 1 targetted at OPAL
[   15.007353656,5] OCC: All Chip Rdy after 0 ms
[   15.316597355,3] STB: container NOT VERIFIED, resource_id=4 secureboot not yet initialized
[   16.434370588,3] STB: container NOT VERIFIED, resource_id=3 secureboot not yet initialized
[   17.113860019,3] CAPP: Error loading ucode lid. index=201d1

Disconnects. Upon reconnection: root@talos:~# obmc-console-client Moving keys from BMC don't give bad pmd anymore. But no luck with USB keyboard still. Manually moving keys through BMC to exit in recovery shell:

root@talos:~# obmc-console-client
!!!!! User requested recovery shell
!!!!! Starting recovery shell
~ # dmesg
[    0.000000] dt-cpu-ftrs: setup for ISA 3000
[    0.000000] dt-cpu-ftrs: final cpu/mmu features = 0x0003c86b8f5fb187 0x3c007041
[    0.000000] radix-mmu: Page sizes from device-tree:
[    0.000000] radix-mmu: Page size shift = 12 AP=0x0
[    0.000000] radix-mmu: Page size shift = 16 AP=0x5
[    0.000000] radix-mmu: Page size shift = 21 AP=0x1
[    0.000000] radix-mmu: Page size shift = 30 AP=0x2
[    0.000000] Activating Kernel Userspace Access Prevention
[    0.000000] Activating Kernel Userspace Execution Prevention
[    0.000000] radix-mmu: Mapped 0x0000000000000000-0x0000000002800000 with 2.00 MiB pages (exec)
[    0.000000] radix-mmu: Mapped 0x0000000002800000-0x0000000040000000 with 2.00 MiB pages
[    0.000000] radix-mmu: Mapped 0x0000000040000000-0x00000000c0000000 with 1.00 GiB pages
[    0.000000] radix-mmu: Mapped 0x00000000c0000000-0x00000000ffe00000 with 2.00 MiB pages
[    0.000000] radix-mmu: Mapped 0x00000000fff00000-0x0000000100000000 with 64.0 KiB pages
[    0.000000] radix-mmu: Mapped 0x0000000100000000-0x00000001c0000000 with 1.00 GiB pages
[    0.000000] radix-mmu: Mapped 0x00000001c0000000-0x00000001ff400000 with 2.00 MiB pages
[    0.000000] radix-mmu: Initializing Radix MMU
[    0.000000] Linux version 6.0.1 (linux-talos-2_workstation.config@linuxboot) (powerpc64le-linux-musl-gcc (GCC) 8.3.0, GNU ld (GNU Binutils) 2.32) #0 SMP 1970-00-00
[    0.000000] OPAL: Found memory mapped LPC bus on chip 0
[    0.000000] ISA: Non-PCI bridge is /lpcm-opb@6030000000000/lpc@0
[    0.000000] Using PowerNV machine description
[    0.000000] printk: bootconsole [udbg0] enabled
[    0.000000] CPU maps initialized for 4 threads per core
[    0.000000]  (thread shift is 2)
[    0.000000] Allocated 3200 bytes for 16 pacas
[    0.000000] -----------------------------------------------------
[    0.000000] phys_mem_size     = 0x1ff300000
[    0.000000] dcache_bsize      = 0x80
[    0.000000] icache_bsize      = 0x80
[    0.000000] cpu_features      = 0x0003c86b8f5fb187
[    0.000000]   possible        = 0x000ffbebcf5fb187
[    0.000000]   always          = 0x0000000380008181
[    0.000000] cpu_user_features = 0xdc0065c2 0xaef00000
[    0.000000] mmu_features      = 0x3c007641
[    0.000000] firmware_features = 0x0000000010000000
[    0.000000] vmalloc start     = 0xc008000000000000
[    0.000000] IO start          = 0xc00a000000000000
[    0.000000] vmemmap start     = 0xc00c000000000000
[    0.000000] -----------------------------------------------------
[    0.000000] numa:   NODE_DATA [mem 0x1ff1aad80-0x1ff1afb7f]
[    0.000000] rfi-flush: fallback displacement flush available
[    0.000000] rfi-flush: patched 10 locations (fallback displacement flush)
[    0.000000] count-cache-flush: flush disabled.
[    0.000000] link-stack-flush: flush disabled.
[    0.000000] entry-flush: patched 59 locations (fallback displacement flush)
[    0.000000] uaccess-flush: patched 1 locations (fallback displacement flush)
[    0.000000] stf-barrier: eieio barrier available
[    0.000000] stf-barrier: patched 59 entry locations (eieio barrier)
[    0.000000] stf-barrier: patched 10 exit locations (eieio barrier)
[    0.000000] OPAL nvram setup, 589824 bytes
[    0.000000] barrier-nospec: using ORI speculation barrier
[    0.000000] barrier-nospec: patched 222 locations
[    0.000000] Top of RAM: 0x1ff400000, Total RAM: 0x1ff300000
[    0.000000] Memory hole size: 1MB
[    0.000000] Zone ranges:
[    0.000000]   Normal   [mem 0x0000000000000000-0x00000001ff3fffff]
[    0.000000] Movable zone start for each node
[    0.000000] Early memory node ranges
[    0.000000]   node   0: [mem 0x0000000000000000-0x00000000ffdfffff]
[    0.000000]   node   0: [mem 0x00000000fff00000-0x00000001ff3fffff]
[    0.000000] Initmem setup node 0 [mem 0x0000000000000000-0x00000001ff3fffff]
[    0.000000] On node 0, zone Normal: 16 pages in unavailable ranges
[    0.000000] On node 0, zone Normal: 192 pages in unavailable ranges
[    0.000000] percpu: Embedded 2 pages/cpu s41256 r0 d89816 u131072
[    0.000000] pcpu-alloc: s41256 r0 d89816 u131072 alloc=2*65536
[    0.000000] pcpu-alloc: [0] 00 [0] 01 [0] 02 [0] 03 [0] 04 [0] 05 [0] 06 [0] 07 
[    0.000000] pcpu-alloc: [0] 08 [0] 09 [0] 10 [0] 11 [0] 12 [0] 13 [0] 14 [0] 15 
[    0.000000] Fallback order for Node 0: 0 
[    0.000000] Built 1 zonelists, mobility grouping on.  Total pages: 130752
[    0.000000] Policy zone: Normal
[    0.000000] Kernel command line: console=tty0 console=hvc0 powersave=off quiet
[    0.000000] Dentry cache hash table entries: 1048576 (order: 7, 8388608 bytes, linear)
[    0.000000] Inode-cache hash table entries: 524288 (order: 6, 4194304 bytes, linear)
[    0.000000] mem auto-init: stack:off, heap alloc:off, heap free:off
[    0.000000] Memory: 8231936K/8375296K available (14848K kernel code, 4864K rwdata, 3008K rodata, 6592K init, 1776K bss, 143360K reserved, 0K cma-reserved)
[    0.000000] SLUB: HWalign=128, Order=0-3, MinObjects=0, CPUs=16, Nodes=1
[    0.000000] rcu: Hierarchical RCU implementation.
[    0.000000] rcu:     RCU event tracing is enabled.
[    0.000000] rcu:     RCU restricting CPUs from NR_CPUS=2048 to nr_cpu_ids=16.
[    0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 10 jiffies.
[    0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=16
[    0.000000] NR_IRQS: 512, nr_irqs: 512, preallocated irqs: 16
[    0.000000] xive: Interrupt handling initialized with native backend
[    0.000000] xive: Using priority 7 for all interrupts
[    0.000000] xive: Using 64kB queues
[    0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.
[    0.000000] time_init: decrementer frequency = 512.000000 MHz
[    0.000000] time_init: processor frequency   = 18446744071914.584320 MHz
[    0.000003] time_init: 56 bit decrementer (max: 7fffffffffffff)
[    0.000007] clocksource: timebase: mask: 0xffffffffffffffff max_cycles: 0x761537d007, max_idle_ns: 440795202126 ns
[    0.000012] clocksource: timebase mult[1f40000] shift[24] registered
[    0.000021] clockevent: decrementer mult[83126f] shift[24] cpu[0]
[    0.000046] random: crng init done
[    0.000109] Console: colour dummy device 80x25
[    0.000148] printk: console [tty0] enabled
[    0.000153] printk: console [hvc0] enabled
[    0.000156] printk: bootconsole [udbg0] disabled
[    0.000198] pid_max: default: 32768 minimum: 301
[    0.000273] LSM: Security Framework initializing
[    0.000320] Mount-cache hash table entries: 16384 (order: 1, 131072 bytes, linear)
[    0.000334] Mountpoint-cache hash table entries: 16384 (order: 1, 131072 bytes, linear)
[    0.000881] POWER9 performance monitor hardware support registered
[    0.000917] rcu: Hierarchical SRCU implementation.
[    0.000919] rcu:     Max phase no-delay instances is 1000.
[    0.001134] smp: Bringing up secondary CPUs ...
[    0.005632] smp: Brought up 1 node, 16 CPUs
[    0.005638] numa: Node 0 CPUs: 0-15
[    0.006527] devtmpfs: initialized
[    0.014372] Initializing IODA2 PHB (/pciex@600c3c0000000)
[    0.014417] PCI host bridge /pciex@600c3c0000000 (primary) ranges:
[    0.014430]  MEM 0x000600c000000000..0x000600c07ffeffff -> 0x0000000080000000 
[    0.014592]  MEM 0x0006000000000000..0x0006003fffffffff -> 0x0006000000000000 (M64 #1..31)
[    0.014598]  Using M64 #31 as default window
[    0.014622]   512 (511) PE's M32: 0x80000000 [segment=0x400000]
[    0.014627]                  M64: 0x4000000000 [segment=0x20000000]
[    0.014632]   Allocated bitmap for 4088 MSIs (base IRQ 0xfe000)
[    0.015177] Initializing IODA2 PHB (/pciex@600c3c0100000)
[    0.015237] PCI host bridge /pciex@600c3c0100000  ranges:
[    0.015247]  MEM 0x000600c080000000..0x000600c0fffeffff -> 0x0000000080000000 
[    0.015406]  MEM 0x0006004000000000..0x0006007fffffffff -> 0x0006004000000000 (M64 #1..15)
[    0.015411]  Using M64 #15 as default window
[    0.015428]   256 (255) PE's M32: 0x80000000 [segment=0x800000]
[    0.015432]                  M64: 0x4000000000 [segment=0x40000000]
[    0.015436]   Allocated bitmap for 2040 MSIs (base IRQ 0xfd800)
[    0.015798] Initializing IODA2 PHB (/pciex@600c3c0200000)
[    0.015857] PCI host bridge /pciex@600c3c0200000  ranges:
[    0.015868]  MEM 0x000600c100000000..0x000600c17ffeffff -> 0x0000000080000000 
[    0.016025]  MEM 0x0006008000000000..0x000600bfffffffff -> 0x0006008000000000 (M64 #1..15)
[    0.016030]  Using M64 #15 as default window
[    0.016044]   256 (255) PE's M32: 0x80000000 [segment=0x800000]
[    0.016048]                  M64: 0x4000000000 [segment=0x40000000]
[    0.016052]   Allocated bitmap for 2040 MSIs (base IRQ 0xfd000)
[    0.016412] Initializing IODA2 PHB (/pciex@600c3c0300000)
[    0.016471] PCI host bridge /pciex@600c3c0300000  ranges:
[    0.016481]  MEM 0x000600c180000000..0x000600c1fffeffff -> 0x0000000080000000 
[    0.016640]  MEM 0x0006020000000000..0x0006023fffffffff -> 0x0006020000000000 (M64 #1..31)
[    0.016645]  Using M64 #31 as default window
[    0.016667]   512 (511) PE's M32: 0x80000000 [segment=0x400000]
[    0.016671]                  M64: 0x4000000000 [segment=0x20000000]
[    0.016675]   Allocated bitmap for 4088 MSIs (base IRQ 0xfc000)
[    0.017212] Initializing IODA2 PHB (/pciex@600c3c0400000)
[    0.017272] PCI host bridge /pciex@600c3c0400000  ranges:
[    0.017282]  MEM 0x000600c200000000..0x000600c27ffeffff -> 0x0000000080000000 
[    0.017440]  MEM 0x0006024000000000..0x0006027fffffffff -> 0x0006024000000000 (M64 #1..15)
[    0.017445]  Using M64 #15 as default window
[    0.017462]   256 (255) PE's M32: 0x80000000 [segment=0x800000]
[    0.017466]                  M64: 0x4000000000 [segment=0x40000000]
[    0.017470]   Allocated bitmap for 2040 MSIs (base IRQ 0xfb800)
[    0.017833] Initializing IODA2 PHB (/pciex@600c3c0500000)
[    0.017892] PCI host bridge /pciex@600c3c0500000  ranges:
[    0.017902]  MEM 0x000600c280000000..0x000600c2fffeffff -> 0x0000000080000000 
[    0.018061]  MEM 0x0006028000000000..0x000602bfffffffff -> 0x0006028000000000 (M64 #1..15)
[    0.018066]  Using M64 #15 as default window
[    0.018083]   256 (255) PE's M32: 0x80000000 [segment=0x800000]
[    0.018087]                  M64: 0x4000000000 [segment=0x40000000]
[    0.018091]   Allocated bitmap for 2040 MSIs (base IRQ 0xfb000)
[    0.018605] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns
[    0.018614] futex hash table entries: 4096 (order: 3, 524288 bytes, linear)
[    0.018856] NET: Registered PF_NETLINK/PF_ROUTE protocol family
[    0.019155] cpuidle: using governor menu
[    0.019901] EEH: PowerNV platform initialized
[    0.024623] PCI: Probing PCI hardware
[    0.024750] PCI host bridge to bus 0000:00
[    0.024756] pci_bus 0000:00: root bus resource [mem 0x600c000000000-0x600c07ffeffff] (bus address [0x80000000-0xfffeffff])
[    0.024763] pci_bus 0000:00: root bus resource [mem 0x6000000000000-0x6003fbfffffff 64bit pref]
[    0.024769] pci_bus 0000:00: root bus resource [bus 00-ff]
[    0.024775] pci_bus 0000:00: busn_res: [bus 00-ff] end is updated to ff
[    0.024822] pci 0000:00:00.0: [1014:04c1] type 01 class 0x060400
[    0.025041] pci 0000:00:00.0: PME# supported from D0 D3hot D3cold
[    0.028471] pci 0000:00:00.0: PCI bridge to [bus 01-ff]
[    0.028513] pci_bus 0000:00: busn_res: [bus 00-ff] end is updated to ff
[    0.028629] PCI host bridge to bus 0001:00
[    0.028634] pci_bus 0001:00: root bus resource [mem 0x600c080000000-0x600c0fffeffff] (bus address [0x80000000-0xfffeffff])
[    0.028639] pci_bus 0001:00: root bus resource [mem 0x6004000000000-0x6007f7fffffff 64bit pref]
[    0.028645] pci_bus 0001:00: root bus resource [bus 00-ff]
[    0.028650] pci_bus 0001:00: busn_res: [bus 00-ff] end is updated to ff
[    0.028686] pci 0001:00:00.0: [1014:04c1] type 01 class 0x060400
[    0.028897] pci 0001:00:00.0: PME# supported from D0 D3hot D3cold
[    0.032316] pci 0001:00:00.0: PCI bridge to [bus 01-ff]
[    0.032357] pci_bus 0001:00: busn_res: [bus 00-ff] end is updated to ff
[    0.032471] PCI host bridge to bus 0002:00
[    0.032475] pci_bus 0002:00: root bus resource [mem 0x600c100000000-0x600c17ffeffff] (bus address [0x80000000-0xfffeffff])
[    0.032481] pci_bus 0002:00: root bus resource [mem 0x6008000000000-0x600bf7fffffff 64bit pref]
[    0.032486] pci_bus 0002:00: root bus resource [bus 00-ff]
[    0.032491] pci_bus 0002:00: busn_res: [bus 00-ff] end is updated to ff
[    0.032527] pci 0002:00:00.0: [1014:04c1] type 01 class 0x060400
[    0.032738] pci 0002:00:00.0: PME# supported from D0 D3hot D3cold
[    0.036152] pci 0002:00:00.0: PCI bridge to [bus 01-ff]
[    0.036192] pci_bus 0002:00: busn_res: [bus 00-ff] end is updated to ff
[    0.036318] PCI host bridge to bus 0003:00
[    0.036322] pci_bus 0003:00: root bus resource [mem 0x600c180000000-0x600c1fffeffff] (bus address [0x80000000-0xfffeffff])
[    0.036328] pci_bus 0003:00: root bus resource [mem 0x6020000000000-0x6023fbfffffff 64bit pref]
[    0.036333] pci_bus 0003:00: root bus resource [bus 00-ff]
[    0.036338] pci_bus 0003:00: busn_res: [bus 00-ff] end is updated to ff
[    0.036374] pci 0003:00:00.0: [1014:04c1] type 01 class 0x060400
[    0.036583] pci 0003:00:00.0: PME# supported from D0 D3hot D3cold
[    0.039925] pci 0003:01:00.0: [104c:8241] type 00 class 0x0c0330
[    0.039972] pci 0003:01:00.0: reg 0x10: [mem 0x00000000-0x0000ffff 64bit]
[    0.040001] pci 0003:01:00.0: reg 0x18: [mem 0x00000000-0x00001fff 64bit]
[    0.040077] pci 0003:01:00.0: BAR2 [mem size 0x00002000 64bit]: requesting alignment to 0x10000
[    0.040206] pci 0003:01:00.0: supports D1 D2
[    0.040210] pci 0003:01:00.0: PME# supported from D0 D1 D2 D3hot
[    0.040357] pci 0003:00:00.0: PCI bridge to [bus 01]
[    0.040396] pci_bus 0003:00: busn_res: [bus 00-ff] end is updated to 01
[    0.040532] PCI host bridge to bus 0004:00
[    0.040537] pci_bus 0004:00: root bus resource [mem 0x600c200000000-0x600c27ffeffff] (bus address [0x80000000-0xfffeffff])
[    0.040542] pci_bus 0004:00: root bus resource [mem 0x6024000000000-0x6027f7fffffff 64bit pref]
[    0.040548] pci_bus 0004:00: root bus resource [bus 00-ff]
[    0.040552] pci_bus 0004:00: busn_res: [bus 00-ff] end is updated to ff
[    0.040588] pci 0004:00:00.0: [1014:04c1] type 01 class 0x060400
[    0.040797] pci 0004:00:00.0: PME# supported from D0 D3hot D3cold
[    0.044152] pci 0004:01:00.0: [14e4:1657] type 00 class 0x020000
[    0.044199] pci 0004:01:00.0: reg 0x10: [mem 0x00000000-0x0000ffff 64bit pref]
[    0.044227] pci 0004:01:00.0: reg 0x18: [mem 0x00000000-0x0000ffff 64bit pref]
[    0.044255] pci 0004:01:00.0: reg 0x20: [mem 0x00000000-0x0000ffff 64bit pref]
[    0.044273] pci 0004:01:00.0: reg 0x30: [mem 0x00000000-0x000007ff pref]
[    0.044320] pci 0004:01:00.0: BAR6 [mem size 0x00000800 pref]: requesting alignment to 0x10000
[    0.044506] pci 0004:01:00.0: PME# supported from D0 D3hot D3cold
[    0.044697] pci 0004:01:00.1: [14e4:1657] type 00 class 0x020000
[    0.044744] pci 0004:01:00.1: reg 0x10: [mem 0x00000000-0x0000ffff 64bit pref]
[    0.044772] pci 0004:01:00.1: reg 0x18: [mem 0x00000000-0x0000ffff 64bit pref]
[    0.044801] pci 0004:01:00.1: reg 0x20: [mem 0x00000000-0x0000ffff 64bit pref]
[    0.044819] pci 0004:01:00.1: reg 0x30: [mem 0x00000000-0x000007ff pref]
[    0.044866] pci 0004:01:00.1: BAR6 [mem size 0x00000800 pref]: requesting alignment to 0x10000
[    0.045052] pci 0004:01:00.1: PME# supported from D0 D3hot D3cold
[    0.045192] pci 0004:00:00.0: PCI bridge to [bus 01]
[    0.045231] pci_bus 0004:00: busn_res: [bus 00-ff] end is updated to 01
[    0.045380] PCI host bridge to bus 0005:00
[    0.045384] pci_bus 0005:00: root bus resource [mem 0x600c280000000-0x600c2fffeffff] (bus address [0x80000000-0xfffeffff])
[    0.045390] pci_bus 0005:00: root bus resource [mem 0x6028000000000-0x602bf7fffffff 64bit pref]
[    0.045395] pci_bus 0005:00: root bus resource [bus 00-ff]
[    0.045400] pci_bus 0005:00: busn_res: [bus 00-ff] end is updated to ff
[    0.045436] pci 0005:00:00.0: [1014:04c1] type 01 class 0x060400
[    0.045645] pci 0005:00:00.0: PME# supported from D0 D3hot D3cold
[    0.049006] pci 0005:01:00.0: [1a03:1150] type 01 class 0x060400
[    0.049128] pci 0005:01:00.0: enabling Extended Tags
[    0.049285] pci 0005:01:00.0: supports D1 D2
[    0.049289] pci 0005:01:00.0: PME# supported from D0 D1 D2 D3hot D3cold
[    0.049418] pci 0005:00:00.0: PCI bridge to [bus 01-02]
[    0.049525] pci_bus 0005:02: extended config space not accessible
[    0.049573] pci 0005:02:00.0: [1a03:2000] type 00 class 0x030000
[    0.049610] pci 0005:02:00.0: reg 0x10: [mem 0x00000000-0x00ffffff]
[    0.049629] pci 0005:02:00.0: reg 0x14: [mem 0x00000000-0x0001ffff]
[    0.049648] pci 0005:02:00.0: reg 0x18: [io  0x0000-0x007f]
[    0.049794] pci 0005:02:00.0: supports D1 D2
[    0.049798] pci 0005:02:00.0: PME# supported from D0 D1 D2 D3hot D3cold
[    0.053040] pci 0005:01:00.0: PCI bridge to [bus 02]
[    0.053101] pci_bus 0005:00: busn_res: [bus 00-ff] end is updated to 02
[    0.053117] pci 0000:00:00.0: PCI bridge to [bus 01-ff]
[    0.053141] pci_bus 0000:00: resource 4 [mem 0x600c000000000-0x600c07ffeffff]
[    0.053146] pci_bus 0000:00: resource 5 [mem 0x6000000000000-0x6003fbfffffff 64bit pref]
[    0.053153] pci 0001:00:00.0: PCI bridge to [bus 01-ff]
[    0.053175] pci_bus 0001:00: resource 4 [mem 0x600c080000000-0x600c0fffeffff]
[    0.053181] pci_bus 0001:00: resource 5 [mem 0x6004000000000-0x6007f7fffffff 64bit pref]
[    0.053187] pci 0002:00:00.0: PCI bridge to [bus 01-ff]
[    0.053209] pci_bus 0002:00: resource 4 [mem 0x600c100000000-0x600c17ffeffff]
[    0.053214] pci_bus 0002:00: resource 5 [mem 0x6008000000000-0x600bf7fffffff 64bit pref]
[    0.053224] pci 0003:00:00.0: BAR 8: assigned [mem 0x600c180000000-0x600c1803fffff]
[    0.053232] pci 0003:01:00.0: BAR 0: assigned [mem 0x600c180000000-0x600c18000ffff 64bit]
[    0.053256] pci 0003:01:00.0: BAR 2: assigned [mem 0x600c180010000-0x600c180011fff 64bit]
[    0.053279] pci 0003:00:00.0: PCI bridge to [bus 01]
[    0.053289] pci 0003:00:00.0:   bridge window [mem 0x600c180000000-0x600c1ffefffff]
[    0.053306] pci_bus 0003:00: resource 4 [mem 0x600c180000000-0x600c1fffeffff]
[    0.053311] pci_bus 0003:00: resource 5 [mem 0x6020000000000-0x6023fbfffffff 64bit pref]
[    0.053316] pci_bus 0003:01: resource 1 [mem 0x600c180000000-0x600c1ffefffff]
[    0.053324] pci 0004:00:00.0: BAR 9: assigned [mem 0x6024000000000-0x602403fffffff 64bit pref]
[    0.053330] pci 0004:00:00.0: BAR 8: assigned [mem 0x600c200000000-0x600c2007fffff]
[    0.053337] pci 0004:01:00.0: BAR 0: assigned [mem 0x6024000000000-0x602400000ffff 64bit pref]
[    0.053361] pci 0004:01:00.0: BAR 2: assigned [mem 0x6024000010000-0x602400001ffff 64bit pref]
[    0.053384] pci 0004:01:00.0: BAR 4: assigned [mem 0x6024000020000-0x602400002ffff 64bit pref]
[    0.053407] pci 0004:01:00.0: BAR 6: assigned [mem 0x600c200000000-0x600c2000007ff pref]
[    0.053413] pci 0004:01:00.1: BAR 0: assigned [mem 0x6024000030000-0x602400003ffff 64bit pref]
[    0.053436] pci 0004:01:00.1: BAR 2: assigned [mem 0x6024000040000-0x602400004ffff 64bit pref]
[    0.053459] pci 0004:01:00.1: BAR 4: assigned [mem 0x6024000050000-0x602400005ffff 64bit pref]
[    0.053482] pci 0004:01:00.1: BAR 6: assigned [mem 0x600c200010000-0x600c2000107ff pref]
[    0.053488] pci 0004:00:00.0: PCI bridge to [bus 01]
[    0.053499] pci 0004:00:00.0:   bridge window [mem 0x600c200000000-0x600c27fefffff]
[    0.053508] pci 0004:00:00.0:   bridge window [mem 0x6024000000000-0x6027f7ff0ffff 64bit pref]
[    0.053521] pci_bus 0004:00: resource 4 [mem 0x600c200000000-0x600c27ffeffff]
[    0.053526] pci_bus 0004:00: resource 5 [mem 0x6024000000000-0x6027f7fffffff 64bit pref]
[    0.053531] pci_bus 0004:01: resource 1 [mem 0x600c200000000-0x600c27fefffff]
[    0.053536] pci_bus 0004:01: resource 2 [mem 0x6024000000000-0x6027f7ff0ffff 64bit pref]
[    0.053545] pci 0005:00:00.0: BAR 8: assigned [mem 0x600c280000000-0x600c2817fffff]
[    0.053550] pci 0005:00:00.0: BAR 7: no space for [io  size 0x1000]
[    0.053555] pci 0005:00:00.0: BAR 7: failed to assign [io  size 0x1000]
[    0.053561] pci 0005:01:00.0: BAR 8: assigned [mem 0x600c280000000-0x600c2817fffff]
[    0.053566] pci 0005:01:00.0: BAR 7: no space for [io  size 0x1000]
[    0.053570] pci 0005:01:00.0: BAR 7: failed to assign [io  size 0x1000]
[    0.053576] pci 0005:02:00.0: BAR 0: assigned [mem 0x600c280000000-0x600c280ffffff]
[    0.053587] pci 0005:02:00.0: BAR 1: assigned [mem 0x600c281000000-0x600c28101ffff]
[    0.053598] pci 0005:02:00.0: BAR 2: no space for [io  size 0x0080]
[    0.053602] pci 0005:02:00.0: BAR 2: failed to assign [io  size 0x0080]
[    0.053608] pci 0005:01:00.0: PCI bridge to [bus 02]
[    0.053618] pci 0005:01:00.0:   bridge window [mem 0x600c280000000-0x600c2ffefffff]
[    0.053636] pci 0005:00:00.0: PCI bridge to [bus 01-02]
[    0.053646] pci 0005:00:00.0:   bridge window [mem 0x600c280000000-0x600c2ffefffff]
[    0.053663] pci_bus 0005:00: Some PCI device resources are unassigned, try booting with pci=realloc
[    0.053667] pci_bus 0005:00: resource 4 [mem 0x600c280000000-0x600c2fffeffff]
[    0.053672] pci_bus 0005:00: resource 5 [mem 0x6028000000000-0x602bf7fffffff 64bit pref]
[    0.053677] pci_bus 0005:01: resource 1 [mem 0x600c280000000-0x600c2ffefffff]
[    0.053682] pci_bus 0005:02: resource 1 [mem 0x600c280000000-0x600c2ffefffff]
[    0.053688] pci_bus 0000:00: Configuring PE for bus
[    0.053692] pci 0000:00     : [PE# 1fe] Secondary bus 0x0000000000000000 associated with PE#1fe
[    0.053958] pci 0000:00:00.0: Configured PE#1fe
[    0.054013] pci_bus 0001:00: Configuring PE for bus
[    0.054017] pci 0001:00     : [PE# fe] Secondary bus 0x0000000000000000 associated with PE#fe
[    0.054279] pci 0001:00:00.0: Configured PE#fe
[    0.054327] pci_bus 0002:00: Configuring PE for bus
[    0.054331] pci 0002:00     : [PE# fe] Secondary bus 0x0000000000000000 associated with PE#fe
[    0.054591] pci 0002:00:00.0: Configured PE#fe
[    0.054638] pci_bus 0003:00: Configuring PE for bus
[    0.054642] pci 0003:00     : [PE# 1fe] Secondary bus 0x0000000000000000 associated with PE#1fe
[    0.054904] pci 0003:00:00.0: Configured PE#1fe
[    0.054951] pci_bus 0003:01: Configuring PE for bus
[    0.054958] pci 0003:01     : [PE# 1fd] Secondary bus 0x0000000000000001 associated with PE#1fd
[    0.055222] pci 0003:01:00.0: Configured PE#1fd
[    0.055226] pci 0003:01     : [PE# 1fd] Setting up 32-bit TCE table at 0..80000000
[    0.056248] IOMMU table initialized, virtual merging enabled
[    0.056253] pci 0003:01     : [PE# 1fd] Setting up window#0 0..ffffffff pg=10000
[    0.056263] pci 0003:01     : [PE# 1fd] Enabling 64-bit DMA bypass
[    0.056282] pci 0003:01:00.0: Adding to iommu group 0
[    0.056387] pci_bus 0004:00: Configuring PE for bus
[    0.056391] pci 0004:00     : [PE# fe] Secondary bus 0x0000000000000000 associated with PE#fe
[    0.056651] pci 0004:00:00.0: Configured PE#fe
[    0.056699] pci_bus 0004:01: Configuring PE for bus
[    0.056715] pci 0004:01     : [PE# 00] Secondary bus 0x0000000000000001 associated with PE#0
[    0.056978] pci 0004:01:00.0: Configured PE#0
[    0.056982] pci 0004:01     : [PE# 00] Setting up 32-bit TCE table at 0..80000000
[    0.057999] pci 0004:01     : [PE# 00] Setting up window#0 0..ffffffff pg=10000
[    0.058008] pci 0004:01     : [PE# 00] Enabling 64-bit DMA bypass
[    0.058023] pci 0004:01:00.0: Adding to iommu group 1
[    0.058139] pci 0004:01:00.1: Added to existing PE#0
[    0.058147] pci 0004:01:00.1: Adding to iommu group 1
[    0.058261] pci_bus 0005:00: Configuring PE for bus
[    0.058265] pci 0005:00     : [PE# fe] Secondary bus 0x0000000000000000 associated with PE#fe
[    0.058526] pci 0005:00:00.0: Configured PE#fe
[    0.058573] pci_bus 0005:01: Configuring PE for bus
[    0.058579] pci 0005:01     : [PE# fd] Secondary bus 0x0000000000000001 associated with PE#fd
[    0.058842] pci 0005:01:00.0: Configured PE#fd
[    0.058943] pci_bus 0005:02: Configuring PE for bus
[    0.058950] pci 0005:02     : [PE# fc] Secondary bus 0x0000000000000002..0x0000000000000002 associated with PE#fc
[    0.059214] pci 0005:02:00.0: Configured PE#fc
[    0.059218] pci 0005:02     : [PE# fc] Setting up 32-bit TCE table at 0..80000000
[    0.060245] pci 0005:02     : [PE# fc] Setting up window#0 0..ffffffff pg=10000
[    0.060254] pci 0005:02     : [PE# fc] Enabling 64-bit DMA bypass
[    0.060270] pci 0005:02:00.0: Adding to iommu group 2
[    0.060390] pci 0003:00:00.0: enabling device (0105 -> 0107)
[    0.060407] pci 0004:00:00.0: enabling device (0105 -> 0107)
[    0.060423] pci 0005:00:00.0: enabling device (0105 -> 0107)
[    0.060441] pci 0005:01:00.0: enabling device (0145 -> 0147)
[    0.060456] EEH: Capable adapter found: recovery enabled.
[    0.060524] PCI: Probing PCI hardware done
[    0.230319] raid6: vpermxor8 gen()  6367 MB/s
[    0.400433] raid6: vpermxor4 gen()  5866 MB/s
[    0.570491] raid6: vpermxor2 gen()  4884 MB/s
[    0.740547] raid6: vpermxor1 gen()  3871 MB/s
[    0.910637] raid6: altivecx8 gen()  3869 MB/s
[    1.080722] raid6: altivecx4 gen()  3890 MB/s
[    1.250845] raid6: altivecx2 gen()  3367 MB/s
[    1.420892] raid6: altivecx1 gen()  2810 MB/s
[    1.591096] raid6: int64x8  gen()  1870 MB/s
[    1.761133] raid6: int64x4  gen()  2156 MB/s
[    1.931193] raid6: int64x2  gen()  1879 MB/s
[    2.101384] raid6: int64x1  gen()  1521 MB/s
[    2.101388] raid6: using algorithm vpermxor8 gen() 6367 MB/s
[    2.101392] raid6: using intx1 recovery algorithm
[    2.101545] iommu: Default domain type: Translated 
[    2.101549] iommu: DMA domain TLB invalidation policy: strict mode 
[    2.101663] SCSI subsystem initialized
[    2.101719] libata version 3.00 loaded.
[    2.101752] usbcore: registered new interface driver usbfs
[    2.101770] usbcore: registered new interface driver hub
[    2.101789] usbcore: registered new device driver usb
[    2.102091] pci 0005:02:00.0: vgaarb: setting as boot VGA device
[    2.102096] pci 0005:02:00.0: vgaarb: bridge control possible
[    2.102100] pci 0005:02:00.0: vgaarb: VGA device added: decodes=io+mem,owns=none,locks=none
[    2.102105] vgaarb: loaded
[    2.102207] clocksource: Switched to clocksource timebase
[    2.102477] VFS: Disk quotas dquot_6.6.0
[    2.102526] VFS: Dquot-cache hash table entries: 8192 (order 0, 65536 bytes)
[    2.104785] NET: Registered PF_INET protocol family
[    2.104892] IP idents hash table entries: 131072 (order: 4, 1048576 bytes, linear)
[    2.108593] tcp_listen_portaddr_hash hash table entries: 4096 (order: 0, 65536 bytes, linear)
[    2.108621] Table-perturb hash table entries: 65536 (order: 2, 262144 bytes, linear)
[    2.108628] TCP established hash table entries: 65536 (order: 3, 524288 bytes, linear)
[    2.108933] TCP bind hash table entries: 65536 (order: 4, 1048576 bytes, linear)
[    2.109208] TCP: Hash tables configured (established 65536 bind 65536)
[    2.109252] UDP hash table entries: 4096 (order: 1, 131072 bytes, linear)
[    2.109303] UDP-Lite hash table entries: 4096 (order: 1, 131072 bytes, linear)
[    2.109396] NET: Registered PF_UNIX/PF_LOCAL protocol family
[    2.109625] RPC: Registered named UNIX socket transport module.
[    2.109630] RPC: Registered udp transport module.
[    2.109634] RPC: Registered tcp transport module.
[    2.109637] RPC: Registered tcp NFSv4.1 backchannel transport module.
[    2.109672] pci 0003:01:00.0: enabling device (0140 -> 0142)
[    2.109731] PCI: CLS 0 bytes, default 128
[    2.129604] Initialise system trusted keyrings
[    2.129659] workingset: timestamp_bits=54 max_order=17 bucket_order=0
[    2.132513] squashfs: version 4.0 (2009/01/31) Phillip Lougher
[    2.132765] NFS: Registering the id_resolver key type
[    2.132773] Key type id_resolver registered
[    2.132776] Key type id_legacy registered
[    2.132814] nfs4filelayout_init: NFSv4 File Layout Driver Registering...
[    2.132817] nfs4flexfilelayout_init: NFSv4 Flexfile Layout Driver Registering...
[    2.132830] jffs2: version 2.2. (NAND) © 2001-2006 Red Hat, Inc.
[    2.133028] JFS: nTxBlock = 4019, nTxLock = 32156
[    2.134198] SGI XFS with ACLs, security attributes, realtime, quota, fatal assert, debug enabled
[    2.134895] F2FS not supported on PAGE_SIZE(65536) != 4096
[    2.134984] xor: measuring software checksum speed
[    2.136901]    8regs           :  5141 MB/sec
[    2.139437]    8regs_prefetch  :  3887 MB/sec
[    2.141333]    32regs          :  5198 MB/sec
[    2.143668]    32regs_prefetch :  4232 MB/sec
[    2.145258]    altivec         :  6198 MB/sec
[    2.145261] xor: using function: altivec (6198 MB/sec)
[    2.145266] Key type asymmetric registered
[    2.145270] Asymmetric key parser 'x509' registered
[    2.145327] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 252)
[    2.146011] IPMI message handler: version 39.2
[    2.146025] ipmi device interface
[    2.159806] ipmi-powernv ibm,opal:ipmi: IPMI message handler: The GUID response from the BMC was too short, it was 1 but should have been 17.  Assuming GUID is not available.
[    2.174408] ipmi-powernv ibm,opal:ipmi: IPMI message handler: Found new BMC (man_id: 0x00cb06, prod_id: 0x0001, dev_id: 0x00)
[    2.277478] hvc0: raw protocol on /ibm,opal/consoles/serial@0 (boot console)
[    2.277508] hvc0: No interrupts property, using OPAL event
[    2.277821] Serial: 8250/16550 driver, 4 ports, IRQ sharing disabled
[    2.278170] Non-volatile memory driver v1.3
[    2.278217] [drm] radeon kernel modesetting enabled.
[    2.278332] ast 0005:02:00.0: enabling device (0140 -> 0142)
[    2.278390] ast 0005:02:00.0: [drm] platform has no IO space, trying MMIO
[    2.278396] ast 0005:02:00.0: [drm] VGA not enabled on entry, requesting chip POST
[    2.278401] ast 0005:02:00.0: [drm] Using device-tree for configuration
[    2.278405] ast 0005:02:00.0: [drm] AST 2500 detected
[    2.278411] ast 0005:02:00.0: [drm] Using analog VGA
[    2.278416] ast 0005:02:00.0: [drm] dram MCLK=800 Mhz type=7 bus_width=16
[    2.286613] [drm] Initialized ast 0.1.0 20120228 for 0005:02:00.0 on minor 0
[    2.872789] Console: switching to colour frame buffer device 240x67
[    2.879478] ast 0005:02:00.0: [drm] fb0: astdrmfb frame buffer device
[    2.925106] brd: module loaded
[    2.927874] loop: module loaded
[    2.927907] st: Version 20160209, fixed bufsize 32768, s/g segs 256
[    3.062304] 5 fixed-partitions partitions found on MTD device flash@0
[    3.062312] Creating 5 MTD partitions on "flash@0":
[    3.062316] 0x000000000000-0x000004000000 : "PNOR"
[    3.062786] 0x0000022a1000-0x000002f39068 : "BOOTKERNEL"
[    3.063221] 0x000003944000-0x000003946000 : "VERSION"
[    3.063675] 0x000003989000-0x0000039c9000 : "IMA_CATALOG"
[    3.064134] 0x000003e10000-0x000003ff0000 : "BOOTKERNFW"
[    3.064672] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
[    3.064678] ohci-pci: OHCI PCI platform driver
[    3.092277] rtc-opal opal-rtc: registered as rtc0
[    3.122230] rtc-opal opal-rtc: setting system clock to 2022-10-14T17:52:27 UTC (1665769947)
[    3.122256] i2c_dev: i2c /dev entries driver
[    3.124781] powernv-cpufreq: cpufreq pstate min 0x76 nominal 0x4e max 0x0
[    3.124785] powernv-cpufreq: Workload Optimized Frequency is enabled in the platform
[    3.125141] ipip: IPv4 and MPLS over IPv4 tunneling driver
[    3.125336] NET: Registered PF_PACKET protocol family
[    3.125389] NET: Registered PF_RXRPC protocol family
[    3.125393] Key type rxrpc registered
[    3.125396] Key type rxrpc_s registered
[    3.125413] Key type dns_resolver registered
[    3.125474] drmem: No dynamic reconfiguration memory found
[    3.126489] powernv-cpufreq: Frequency Control disabled from OS
[    3.126492] powernv-cpufreq: PMSR = 4e4e763080000000
[    3.126496] powernv-cpufreq: CPU Frequency could be throttled
[    3.126536] registered taskstats version 1
[    3.126540] Loading compiled-in X.509 certificates
[    3.128572] Loaded X.509 cert 'Build time autogenerated kernel key: 0e3bd492ff5ebb9d190d8c85dc612715d0e1e51d'
[    3.128581] kAFS: Red Hat AFS client v0.1 registering.
[    3.322210] kAFS: failed to register: -97
[    3.322797] Btrfs loaded, crc32c=crc32c-generic, zoned=no, fsverity=no
[    3.323253] Key type encrypted registered
[    3.323283] ima: No TPM chip found, activating TPM-bypass!
[    3.323329] ima: Allocated hash algorithm: sha1
[    3.323345] ima: No architecture policies found
[    3.323366] evm: Initialising EVM extended attributes:
[    3.323369] evm: security.selinux (disabled)
[    3.323372] evm: security.SMACK64 (disabled)
[    3.323374] evm: security.SMACK64EXEC (disabled)
[    3.323377] evm: security.SMACK64TRANSMUTE (disabled)
[    3.323380] evm: security.SMACK64MMAP (disabled)
[    3.323383] evm: security.apparmor (disabled)
[    3.323385] evm: security.ima (disabled)
[    3.323388] evm: security.capability
[    3.323391] evm: HMAC attrs: 0x1
[    3.325140] Freeing unused kernel image (initmem) memory: 6592K
[    3.442259] Run /init as init process
[    3.442262]   with arguments:
[    3.442265]     /init
[    3.442267]   with environment:
[    3.442270]     HOME=/
[    3.442272]     TERM=linux
[    3.443917] [U] hello world
[   90.690887] ehci_hcd: module verification failed: signature and/or required key missing - tainting kernel
[   90.695210] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[   90.695213] Warning! ehci_hcd should always be loaded before uhci_hcd and ohci_hcd, not after
[   90.698994] ehci-pci: EHCI PCI platform driver
[   90.717815] xhci_hcd 0003:01:00.0: xHCI Host Controller
[   90.717900] xhci_hcd 0003:01:00.0: new USB bus registered, assigned bus number 1
[   90.718041] xhci_hcd 0003:01:00.0: hcc params 0x0270f06d hci version 0x96 quirks 0x0000000004000000
[   90.718560] xhci_hcd 0003:01:00.0: xHCI Host Controller
[   90.718607] xhci_hcd 0003:01:00.0: new USB bus registered, assigned bus number 2
[   90.718614] xhci_hcd 0003:01:00.0: Host supports USB 3.0 SuperSpeed
[   90.718854] hub 1-0:1.0: USB hub found
[   90.718871] hub 1-0:1.0: 4 ports detected
[   90.719073] usb usb2: We don't know the algorithms for LPM for this host, disabling LPM.
[   90.719250] hub 2-0:1.0: USB hub found
[   90.719265] hub 2-0:1.0: 4 ports detected
[   91.012218] usb 1-1: new high-speed USB device number 2 using xhci_hcd
[   91.207162] hub 1-1:1.0: USB hub found
[   91.207760] hub 1-1:1.0: 4 ports detected
[   91.372222] usb 1-2: new high-speed USB device number 3 using xhci_hcd
[   91.652212] usb 1-1.1: new low-speed USB device number 4 using xhci_hcd
[   91.932207] usb 1-3: new high-speed USB device number 5 using xhci_hcd
[   92.127137] hub 1-3:1.0: USB hub found
[   92.127484] hub 1-3:1.0: 5 ports detected
[   92.212210] usb 1-1.3: new low-speed USB device number 6 using xhci_hcd
[   92.492217] usb 1-4: new high-speed USB device number 7 using xhci_hcd
[   92.687523] hub 1-4:1.0: USB hub found
[   92.688126] hub 1-4:1.0: 4 ports detected
[   92.734187] input: LOGITECH G110 G-keys Keypad as /devices/pci0003:00/0003:00:00.0/0003:01:00.0/usb1/1-1/1-1.1/1-1.1:1.0/0003:046D:C22B.0001/input/input0
[   92.772211] usb 1-3.1: new high-speed USB device number 8 using xhci_hcd
[   92.802324] hid-generic 0003:046D:C22B.0001: input,hiddev96: USB HID v1.00 Keypad [LOGITECH G110 G-keys] on usb-0003:01:00.0-1.1/input0
[   92.802358] usbhid 1-1.1:1.1: couldn't find an input interrupt endpoint
[   92.806077] input: Gaming Keyboard G110 as /devices/pci0003:00/0003:00:00.0/0003:01:00.0/usb1/1-1/1-1.3/1-1.3:1.0/0003:046D:C22A.0002/input/input2
[   92.872292] hid-generic 0003:046D:C22A.0002: input: USB HID v1.10 Keyboard [Gaming Keyboard G110] on usb-0003:01:00.0-1.3/input0
[   92.876878] input: Gaming Keyboard G110 Consumer Control as /devices/pci0003:00/0003:00:00.0/0003:01:00.0/usb1/1-1/1-1.3/1-1.3:1.1/0003:046D:C22A.0003/input/input3
[   92.942301] hid-generic 0003:046D:C22A.0003: input,hiddev97: USB HID v1.10 Device [Gaming Keyboard G110] on usb-0003:01:00.0-1.3/input1
[   92.942344] usbcore: registered new interface driver usbhid
[   92.942348] usbhid: USB HID core driver
[   92.951222] input: OpenBMC virtual_input as /devices/pci0003:00/0003:00:00.0/0003:01:00.0/usb1/1-3/1-3.1/1-3.1:1.0/0003:1D6B:0104.0004/input/input5
[   93.012265] hid-generic 0003:1D6B:0104.0004: input: USB HID v1.01 Keyboard [OpenBMC virtual_input] on usb-0003:01:00.0-3.1/input0
[   93.014179] input: OpenBMC virtual_input as /devices/pci0003:00/0003:00:00.0/0003:01:00.0/usb1/1-3/1-3.1/1-3.1:1.1/0003:1D6B:0104.0005/input/input6
[   93.014228] hid-generic 0003:1D6B:0104.0005: input: USB HID v1.01 Mouse [OpenBMC virtual_input] on usb-0003:01:00.0-3.1/input1
[   93.032212] usb 1-4.3: new high-speed USB device number 9 using xhci_hcd
[   93.226104] hid-generic 0003:20A0:4109.0006: hiddev98: USB HID v1.10 Device [Nitrokey Nitrokey Storage] on usb-0003:01:00.0-4.3/input2
~ # 
~ # cbmem -1

coreboot--TIMELESS--LESSTIME--Heads-v0.2.0-1267-g73bc621 Thu Jan  1 00:00:00 UTC 1970 bootblock starting (log level: 7)...
FFS header at 0x80060300ffff7000
PNOR base at 0x80060300fc000000
HBI partition has ECC
HBI is in 0x00426200 through 0x0175f037
FMAP: Found "FLASH" version 1.1 at 0x20000.
FMAP: base = 0x0 size = 0x200000 #areas = 4
FMAP: area COREBOOT found @ 20200 (1965568 bytes)
CBFS: mcache @0xf8231000 built for 10 files, used 0x1f0 of 0x2000 bytes
CBFS: Found 'fallback/romstage' @0x80 size 0x124a1 in mcache @0xf823102c
BS: bootblock times (exec / console): total (unknown) / 2 ms

coreboot--TIMELESS--LESSTIME--Heads-v0.2.0-1267-g73bc621 Thu Jan  1 00:00:00 UTC 1970 romstage starting (log level: 7)...
IPMI: romstage PNP BT 0xe4
Get BMC self test result...Function Not Implemented
Initializing IPMI BMC watchdog timer
IPMI BMC watchdog initialized and started.
Initializing FSI...
Initialized FSI (chips mask: 0x01)
Building MVPDs...
starting istep 8.1
starting istep 8.2
starting istep 8.3
starting istep 8.4
starting istep 8.9
Base epsilon values read from table:
 R_T[0] = 22
 R_T[1] = 22
 R_T[2] = 103
 W_T[0] = 5
 W_T[1] = 33
Scaled epsilon values based on +20 percent guardband:
 R_T[0] = 27
 R_T[1] = 27
 R_T[2] = 124
 W_T[0] = 6
 W_T[1] = 40
starting istep 8.10
starting istep 8.11
starting istep 9.2
starting istep 9.4
starting istep 9.6
starting istep 9.7
starting istep 10.1
starting istep 10.6
starting istep 10.10
starting istep 10.12
starting istep 10.13
FFS header at 0x80060300ffff7000
PNOR base at 0x80060300fc000000
MEMD partition has ECC
MEMD is in 0x03cef200 through 0x03cfb917
I2C transfer failed to complete (0x04011f0104000000)
No memory DIMM at address 51
I2C transfer failed to complete (0x04011f0104000000)
No memory DIMM at address 52
I2C transfer failed to complete (0x04011f0104000000)
No memory DIMM at address 53
I2C transfer failed to complete (0x04011f0104000000)
No memory DIMM at address D4
I2C transfer failed to complete (0x04011f0104000000)
No memory DIMM at address D5
I2C transfer failed to complete (0x04011f0104000000)
No memory DIMM at address D6
I2C transfer failed to complete (0x04011f0104000000)
No memory DIMM at address D7
SPD @ 0x50
SPD: module type is DDR4
SPD: module part number is M393A1K43BB0-CRC    
SPD: banks 16, ranks 1, rows 16, columns 10, density 8192 Mb
SPD: device width 8 bits, bus width 64 bits
SPD: module size is 8192 MB (per channel)
starting istep 13.1
starting istep 13.2
starting istep 13.3
starting istep 13.4
starting istep 13.5
starting istep 13.6
starting istep 13.7
starting istep 13.8
starting istep 13.9
starting istep 13.10
CCS took 2 us (3 us timeout), 1 instruction(s)
CCS took 2 us (2 us timeout), 14 instruction(s)
RCD dump for I2C address 0x58:
0xf820fa30: 80 b3 40 42 30 00 00 00 02 01 00 03 cb e3 c0 0d  ..@B0...........
0xf820fa40: 00 00 39 00 00 00 00 00 00 00 07 00 00 00 00 00  ..9.............
starting istep 13.11
CCS took 2 us (7 us timeout), 2 instruction(s)
Write Leveling starting
CCS took 12 us (92 us timeout), 5 instruction(s)
Write Leveling done
Initial Pattern Write starting
CCS took 5 us (38 us timeout), 5 instruction(s)
Initial Pattern Write done
DQS alignment starting
CCS took 11 us (44 us timeout), 1 instruction(s)
DQS alignment done
Read Clock Alignment starting
CCS took 8 us (82 us timeout), 1 instruction(s)
Read Clock Alignment done
Read Centering starting
CCS took 37 us (120 us timeout), 1 instruction(s)
Read Centering done
Write Centering starting
CCS took 9118 us (11314 us timeout), 7 instruction(s)
Write Centering done
Coarse write/read starting
CCS took 5 us (24 us timeout), 1 instruction(s)
Coarse write/read done
starting istep 13.12
starting istep 13.13
starting istep 14.1
starting istep 14.2
starting istep 14.3
Initializing PEC0...
Initializing PEC1...
Initializing PEC2...
Initializing PHB0...
Initializing PHB1...
Initializing PHB2...
Initializing PHB3...
Initializing PHB4...
Initializing PHB5...
starting istep 14.4
starting istep 14.5
0xF000F = 221d104900008040
CBMEM:
IMD: root @ 0xffeff000 254 entries.
IMD: root @ 0xffefec00 62 entries.
FMAP: area COREBOOT found @ 20200 (1965568 bytes)
FFS header at 0x80060300ffff7000
PNOR base at 0x80060300fc000000
HBI partition has ECC
HBI is in 0x00426200 through 0x0175f037
CBFS: Found 'fallback/ramstage' @0x12580 size 0xcca9 in mcache @0xf8231080
BS: romstage times (exec / console): total (unknown) / 17 ms

coreboot--TIMELESS--LESSTIME--Heads-v0.2.0-1267-g73bc621 Thu Jan  1 00:00:00 UTC 1970 ramstage starting (log level: 7)...
Enumerating buses...
Root Device scanning...
DD21, boot core: 18
FFS header at 0x80060300ffff7000
PNOR base at 0x80060300fc000000
HCODE partition has ECC
HCODE is in 0x01a82200 through 0x01b821ff
FFS header at 0x80060300ffff7000
PNOR base at 0x80060300fc000000
OCC partition has ECC
OCC is in 0x03822200 through 0x039221ff
Base epsilon values read from table:
 R_T[0] = 22
 R_T[1] = 22
 R_T[2] = 103
 W_T[0] = 5
 W_T[1] = 33
Scaled epsilon values based on +20 percent guardband:
 R_T[0] = 27
 R_T[1] = 27
 R_T[2] = 124
 W_T[0] = 6
 W_T[1] = 40
FFS header at 0x80060300ffff7000
PNOR base at 0x80060300fc000000
WOFDATA partition has ECC
WOFDATA is in 0x039ea200 through 0x03c94ca7
Matching WOF tables section not found, disabling WOF
Safe mode freq = 1833412 kHZ, voltage = 711 mv
starting istep 15.2
starting istep 15.3
starting istep 15.4
Starting PM complex...
Attempting PGPE activation...
PGPE was activated successfully
Done starting PM complex
Activating OCC...
Done activating OCC
starting istep 16.1
XIVE configured, entering dead man loop
ASSERTION ERROR: file '(filenames not available on timeless builds)', line 404
starting istep 18.11
starting istep 18.12
CPU_CLUSTER: 0 enabled
PNP: 00e4.0 enabled
scan_bus: bus Root Device finished in 3072 msecs
done
BS: BS_DEV_ENUMERATE run times (exec / console): 3066 / 7 ms
Allocating resources...
Reading resources...
CPU_CLUSTER: 0 missing read_resources
Done reading resources.
CPU_CLUSTER: 0 missing set_resources
PNP: 00e4.0 00 <- [0x00000000e4 - 0x00000000e6] size 0x00000003 gran 0x00 io
Done setting resources.
Done allocating resources.
BS: BS_DEV_RESOURCES run times (exec / console): 0 / 1 ms
Enabling resources...
done.
Initializing devices...
PNP: 00e4.0 init
IPMI: PNP BT 0xe4
Get BMC self test result...Function Not Implemented
IPMI: Found man_id 0x6cb0000, prod_id 0x1000000
IPMI: Version 2.0
PNP: 00e4.0 init finished in 19 msecs
Devices initialized
BS: BS_DEV_INIT run times (exec / console): 19 / 1 ms
Finalize devices...
Devices finalized
Writing coreboot table at 0xffed9000
 0. 0000000000000000-00000000f8207fff: RAM
 1. 00000000f8208000-00000000f820ffff: RAMSTAGE
 2. 00000000f8210000-00000000f8ffffff: RAM
 3. 00000000f9000000-00000000f929bfff: RAMSTAGE
 4. 00000000f929c000-00000000ffed8fff: RAM
 5. 00000000ffed9000-00000000ffefffff: CONFIGURATION TABLES
 6. 00000000fff00000-00000001ff3fffff: RAM
 7. 00000001ff400000-00000001ffffffff: RESERVED
FMAP: area COREBOOT found @ 20200 (1965568 bytes)
FFS header at 0x80060300ffff7000
PNOR base at 0x80060300fc000000
HBI partition has ECC
HBI is in 0x00426200 through 0x0175f037
Wrote coreboot table at: 0xffed9000, 0x214 bytes, checksum 3d45
coreboot table: 556 bytes.
IMD ROOT    0. 0xffeff000 0x00001000
IMD SMALL   1. 0xffefe000 0x00001000
CONSOLE     2. 0xffede000 0x00020000
MEM INFO    3. 0xffedb000 0x000022b0
COREBOOT    4. 0xffed9000 0x00002000
IMD small region:
  IMD ROOT    0. 0xffefec00 0x00000400
  RO MCACHE   1. 0xffefea00 0x000001f0
  FMAP        2. 0xffefe920 0x000000e0
BS: BS_WRITE_TABLES run times (exec / console): 0 / 6 ms
CBFS: Found 'fallback/payload' @0x257c0 size 0x632f9 in mcache @0xffefeb84
Checking segment from ROM address 0xf8380000
Checking segment from ROM address 0xf838001c
Checking segment from ROM address 0xf8380038
Loading segment from ROM address 0xf8380000
  code (compression=1)
  New segment dstaddr 0x00000000 memsize 0x173c00 srcaddr 0xf8380054 filesize 0x632a5
Loading Segment: addr: 0x00000000 memsz: 0x0000000000173c00 filesz: 0x00000000000632a5
using LZMA
Loading segment from ROM address 0xf838001c
  BSS 0x00300000 (113776 byte)
Loading Segment: addr: 0x00300000 memsz: 0x000000000001bc70 filesz: 0x0000000000000000
it's not compressed!
Clearing Segment: addr: 0x0000000000300000 memsz: 0x000000000001bc70
Loading segment from ROM address 0xf8380038
  Entry Point 0x00002570
BS: BS_PAYLOAD_LOAD run times (exec / console): 297 / 4 ms
Jumping to boot code at 0x00002570(0xffed9000)
CBFS: Found '1-cpu.dtb' @0x1f700 size 0x24e7 in mcache @0xffefeb3c

Falling back to server board artifacts of CircleCI to test https://github.com/osresearch/heads/pull/1222

[tlaurion]

Ok, as of current codebase:

init calls cbfs-init, which for talos calls flashrom to get backup of the firmware (which currently reads it 3 times, so around 90 seconds, could be reduced to a minimum of 30) through https://github.com/osresearch/heads/pull/1222/files#diff-0102d43c07e95fc76c3c2fe1532c91a911cabcb96176d0895a937b13c8a7a5f4R10-R23

@SergiiDmytruk : since flashtools is already being modified, wouldn't it make more sense to modify cbfs to do the proper jumps if pnor and extract from ROM directly instead of relaying on flashrom to dump everything at each boot and rely on it (massive TOCTOU here on /tmp/? ) Otherwise, peek and poke from flashtools maybe?

Adding 30 seconds at each boot (if flash.sh is modified to read only once, while write implies a verification) is kinda unacceptable here...

[macpijan]

Going back to the original USB keyboard issue.

The behaviour on Hostboot is the same. So it is not a firmware issue.

We have tried many different console combinations already.

Initially the expectation was that we have a kernel module issue, while we might be hitting some limitation in the kernel console subsystem or need to know a specific workaround for making USB keyboards work when graphics is used.

We are really out of ideas here and we do not see how we can support this from a firmware perspective.

Maybe we should look for help in some POWER9 / Talos II communities, as this problem is not really Dasharo-specific.

[tlaurion]

@miczyg1 @SergiiDmytruk

I just discovered that on the server board, Heads is answering commands through two different terminals, giving whiptail output on two different terminals individually for bmc/host, where the keyboard interacts with the host terminal and where the bmc keyboarfd interacts with bmc terminal only. Not sure why, but on kgpe-d16 board (older kernel as well) that was not possible through ast2050. The BMC and keyboard were interacting with the same terminal.

This is problematic, since a local user could steal unsealed secrets under /tmp from a concurrent session. Heads expects to be single user on the machine...

Any idea where that behavior is coming from? Intuition tells that if we had only one terminal where bmc/usb keyboard was directed to, the problem we have here (and the security issue) would vanish at the same time. Any insight welcome. Test by youself on server board: you can interact in parallel through bmc emulated keyboard-> bmc terminal/usb keyboard->VGA output. Not normal indeed.

Issue discovered trying to tune terminal under https://github.com/osresearch/heads/issues/1237

[SergiiDmytruk]

This is how BMC terminal was made to work, by starting Heads for each console, same as Petitboot does it. I suspect that not passing two consoles to the kernel will just leave you without BMC or normal console and I'm not sure two consoles can be somehow synchronized to act as one (in a screen/tmux-like way).

[tlaurion]

Last console specified has impact for input https://tldp.org/HOWTO/Remote-Serial-Console-HOWTO/configure-kernel.html (hint from @miczyg1 from community call)

Will re-read, had to deal with that with kgpe-d16/qemu boards)

"When multiple consoles are listed output is sent to all consoles and input is taken from the last listed console. The last console is the one Linux uses as the /dev/console device."

Right now default console is setuped to be "console=hvc0" being the last one targeted.

Comparing kgpe-d16 config to talos II, the big difference we can see is that the first console is serial https://github.com/osresearch/heads/blob/fde7ee2b11806b437722f3ccfe1a7f9ff238bb98/boards/kgpe-d16_workstation/kgpe-d16_workstation.config#L60

A dual console input/output is given with qemu board to the payload: https://github.com/osresearch/heads/blob/master/config/coreboot-qemu-fbwhiptail-tpm1-hotp.config#L8

Will try to build and test on kgpe-d16 and check with qemu dual input/output config to validate behavior there as well.

[SergiiDmytruk]

I had to swap the order of consoles in coreboot-qemu-fbwhiptail-tpm1-hotp.config to be able to see Heads via -serial stdio. I think "output is sent to all consoles" refers primarily to dmesg output.

[tlaurion]

I had to swap the order of consoles in coreboot-qemu-fbwhiptail-tpm1-hotp.config to be able to see Heads via -serial stdio. I think "output is sent to all consoles" refers primarily to dmesg output.

No.

Test the expected behavior with the following patch:

diff --git a/boards/qemu-coreboot-fbwhiptail-tpm1-hotp/qemu-coreboot-fbwhiptail-tpm1-hotp.config b/boards/qemu-coreboot-fbwhiptail-tpm1-hotp/qemu-coreboot-fbwhiptail-tpm1-hotp.config
index f7130e0c..28be73ef 100644
--- a/boards/qemu-coreboot-fbwhiptail-tpm1-hotp/qemu-coreboot-fbwhiptail-tpm1-hotp.config
+++ b/boards/qemu-coreboot-fbwhiptail-tpm1-hotp/qemu-coreboot-fbwhiptail-tpm1-hotp.config
@@ -53,6 +53,7 @@ export CONFIG_BOOTSCRIPT=/bin/gui-init
 #export CONFIG_BOOTSCRIPT=/bin/generic-init
 export CONFIG_BOOT_REQ_HASH=n
 export CONFIG_BOOT_REQ_ROLLBACK=n
+export CONFIG_BOOT_RECOVERY_SERIAL="/dev/ttyS0"
 export CONFIG_BOOT_KERNEL_ADD="console=ttyS0 console=tty systemd.zram=0"
 export CONFIG_BOOT_KERNEL_REMOVE="quiet rhgb splash"

diff --git a/boards/qemu-coreboot-whiptail-tpm1/qemu-coreboot-whiptail-tpm1.config b/boards/qemu-coreboot-whiptail-tpm1/qemu-coreboot-whiptail-tpm1.config
index 633aa09e..4b11e94a 100644
--- a/boards/qemu-coreboot-whiptail-tpm1/qemu-coreboot-whiptail-tpm1.config
+++ b/boards/qemu-coreboot-whiptail-tpm1/qemu-coreboot-whiptail-tpm1.config
@@ -51,6 +51,7 @@ export CONFIG_BOOTSCRIPT=/bin/gui-init
 #export CONFIG_BOOTSCRIPT=/bin/generic-init
 export CONFIG_BOOT_REQ_HASH=n
 export CONFIG_BOOT_REQ_ROLLBACK=n
+export CONFIG_BOOT_RECOVERY_SERIAL="/dev/ttyS0"
 export CONFIG_BOOT_KERNEL_ADD="console=ttyS0 console=tty systemd.zram=0"
 export CONFIG_BOOT_KERNEL_REMOVE="quiet rhgb splash"

@SergiiDmytruk : As a result, you will get expected pause_recovery (etc/functions) output "Hit enter to proceed to recovery shell:" prompt on ttyS0 (stdio) while whiptail is shown only once on host (qemu/kvm). This is important in case of TPM, since entering recovery shell will invalidate measurements and prevent TPM disk encryption key to be unsealed.

By removing the CONFIG_BOOT_RECOVERY_SERIAL board configuration export, then agreed, only dmesg is redirected to ttyS0, where again, last console passed per coreboot/skiboot/kernel configured boot option will be /dev/console, so of course switching them results into where the input output is being redirected/taken from.

Same should apply for kgpe-d16 per board configs:

boards/kgpe-d16_server-whiptail/kgpe-d16_server-whiptail.config:export CONFIG_BOOT_RECOVERY_SERIAL="/dev/ttyS0"
boards/kgpe-d16_server/kgpe-d16_server.config:export CONFIG_BOOT_RECOVERY_SERIAL="/dev/ttyS0"

I am still unclear, since I was not able to build coreboot 4.11 for kgpe-d16 (race condition error with car.data at build time...) of what is the behavior there, where on kgpe-d16, ttyS1 is the output for console and where ttyS0 was the possible additional input, I do not recall exactly the details of how the BMC interaction was different and as I told to @miczyg1 in last community meeting, maybe my memory is failing me and the BMC was not able to control host's VGA fbwiptail as it is currently the case for Talos II. Not sure what is exactly the differences at play with hvc0 magic and cannot retest as of now behavior on kgpe-d16.

Can you remind me what was wrong with stty not working for Talos and required agetty and spawning multiple whiptail on each available consoles?

[tlaurion]

kgpe-d16 server:

#CONSOLE SELECTION
#Single output to OpenBMC
export CONFIG_BOOT_KERNEL_ADD="nohz=on console=ttyS1,115200n8 "

#Dual output to local console (tty0) and OpenBmc (ttyS1)
#export CONFIG_BOOT_KERNEL_ADD="nohz=on console=ttyS1,115200n8 console=tty0"

#Single output to tty0
#export CONFIG_BOOT_KERNEL_ADD="nohz=on"
export CONFIG_BOOT_RECOVERY_SERIAL="/dev/ttyS0"

Which tells us that with the help of CONFIG_BOOT_RECOVERY_SERIAL and dual output (tty0) and BMC(ttyS1) it is expected to have dual output and input: export CONFIG_BOOT_KERNEL_ADD="nohz=on console=ttyS1,115200n8 console=tty0

But the question of if remote bmc input was interacting with whiptail or not is not yet clear. @tonux599: can you confirm/infirm?

[SergiiDmytruk]

Can you remind me what was wrong with stty not working for Talos and required agetty and spawning multiple whiptail on each available consoles?

I don't remember trying CONFIG_BOOT_RECOVERY_SERIAL. Petitboot worked on both consoles, so I used it as a reference. From Jira:

The second console is separate from the text one (hvc0 is text and VGA is tty0 and tty1). When petitboot is in use, text and VGA consoles run two independent instances of petitboot. Likely need to do the same for Heads.

bash > /dev/tty0 2>&1 < /dev/tty0 seems to work, but then text console doesn't work even if the command is backgrounded.

[tlaurion]

@SergiiDmytruk things are unfolding themselves with tests of https://app.circleci.com/pipelines/github/tlaurion/heads?branch=talos_CONFIG_BOOT_RECOVERY_SERIAL builds.

First things first, it is impossible to have multiple fbwhiptail output into anything else then the framebuffer itself. So the workstation board config, as proposed in that PR swaps hvc0 and tty0 output, putting tty0 as being the main terminal for console input/output. Doing so, usb keyboard is effective, and interacts with tty0 and through singke fb device. By attemting to spawn fbwhiptail on both tty0 and hvc0 (impossible), it seems that we are confusing input/output, and that the tty0 fb is incorrectly bound, since two fbwhiptail are attempted to be launched.

The problem we have with fbwhiptail as opposed to whiptail output is that even if CONFIG_BOOT_RECOVERY_SERIAL defines hvc0 to spawn a recovery console on hvc0 (bmc) anything that will be drawn on hvc0 outside of console output will attempt to output on fb, which hvc0 can't do.

As in those test roms for workstation, hvc0 will present a console only recovery prompt (asking to type enter on hvc0 to spwan recovery shell and invalidate measurements), where tty0 shows fbwhiptail output.

In TPM usage use case, anything going into the recovery shell will invalidate TPM measurements by extending https://osresearch.net/Keys/#tpm-pcrs PCR4.

Same applies to server board in current testing ROMs from this PR, which shares exactly the same inversed config. Note that the inversion to have tty0 as last defined console was needed to have Fedora's anaconda prompt on screen to start installer as through vnc, otherwise user has to kill anaconda from tty0 terminal to spawn it again, the terminal redirection being lost and having main output attempted to be sent to hvc, which fails (well, waits for input in unbound terminal)

The conclusion here are:

• We cannot start multiple fbwhiptails at the same time if there is only one fb terminal under Heads (the only one valid being on tty0, input/ouput redirection seem to be bound to second spawned fbwhiptail without usb keyboard input being bound to it).
• The current codebase uses the presence of fbwhiptail binary to define terminal output, not the actual console source. Consequently, launching any script which calls whiptail will call fbwhiptail, which cannot draw into hvc0
• We might want to drop fbwhiptail and use whiptail for both server and workstation boards

Unanswered questions:

• Should we have dual whiptail (no-fbwhiptail) output both to hvc0 and tty0 for workstation (as master) server board? By swapping consoles both under Heads linux config and board config, installers work correctly as well.
• Should we have single whiptail output (single console) from server board to hvc0 only and have the recovery shell console on tty0? On server board config, we expect the machine to be remotely.

My security related question before was related to a race condition and security hazard of having multiple consoles under Heads under TPM use case, unsealing secrets under /tmp and having those accessible after having been unsealed from TPM. But to be able to access such files under /tmo, a user would need to access the recovery shell, and launching the recovery shell actually wiped /tmp/secret/* content. All heads related screts are under that directory, including the TPM released disk decryption key that is passed as cpio through kexec-insert-key and kexec-select-boot codepaths. So no issue here into having multiple consoles.

The choice here is user experience based. Basically, my question for the next step of this would be to agree that either:

• [ ] Workstation board config should be using whiptail (not fbwhiptail, just like petitboot output) and output to both hvc0 and tty0 (inverted order on both Heads linux config and board config to be passed to kexec'ed OS), going into recovery from either console invalidating sealed secrets (including TOTP/HOTP/Disk encryption key to be released)
• [ ] Workstation outputs to fbwhiptail on tty0 and through whiptail through hvc0 (fbwhiptail offers code coloring : red background for errors, yellow for warnings, red coloring of consoles passed to OS to remind of unsafe booting. Those are lost through whiptail)
• [ ] Workstation and server boards configs are duplicate. If so, we should offer only a Talos II board config, outputing dual whiptail console output, and dealing independently with bmc input/output and tty0/usb output/input.

Thoughts?

[tlaurion]

Workstation outputs to fbwhiptail on tty0 and through whiptail through hvc0 (fbwhiptail offers code coloring : red background for errors, yellow for warnings, red coloring of consoles passed to OS to remind of unsafe booting. Those are lost through whiptail)

When using fbwhiptail, the content of bin/whiptail is replaced by a shell script that calls fbwhiptail instead, so the rest of the codebase simply calls whiptail, indifferenciating if using whiptail or fbwhiptail... So having both fbwhiptail and whiptail output would require more work.

Basically, if we are to keep a workstation board that uses fbwhiptail today as currently delivered, it simply doesn't work for hvc0 for everything outputted to serial (ttyS0/hvc0). So the get home message here is that fbwhiptail enablement cannot be used in BMC use case as of now.

We could implement a mitigation under bin/whiptail script to simply bail if $(tty) returns anything ttyS*/hv0 prior of attempting to call fbwhiptail as of now.

But as current, fbwhiptail is damn slow (fb is not optimized somehow on used kernel) and screen is somehow corrupted (fuzzy). The good news is that USB keyboard support is now figured out on why it didn't work cannot spawn fbwhptail on hvc0, which is happening now and intercepts usb input somehow).

[SergiiDmytruk]

Nice. Your explanation makes sense and explains why when 2 fbwhiptail were run you could seemingly control BMC's fbwhiptail from the serial and it was still visible after being closed. Given the lags of graphics, I thought it's a bug, but it really was a second instance.

I guess I would pick two text consoles and one board. Doing changes for fbwhiptail doesn't seem to be worth it.

[miczyg1]

Exit criteria:

• have one platform target only (no division on server or workstation)
• have output on both VGA and KVM with whiptail

[tlaurion]

This is fixed under Heads since talos_2 unique board (whiptail) replacing fbwhiptail that was faulty and to be honest too slow to be useable.

https://github.com/osresearch/heads/pull/1353 also fixed an accidental removal of agetty.

I think this issue can be closed.