Feature request : KGPE-D16 in workstation mode with Aspeed supported in FbWhiptail (second GPU HVM passthrough desired under QubesOS) and Pike support

[ghost]

Do i need replace W25Q16BVAIG to W25Q128FVSG for heads flash?

I have built 16MB rom heads firmware using make BOARD=kgpe-d16

user@builder-heads:~/heads/build/kgpe-d16$ ls -lh kgpe-d16.rom 
16M

but W25Q16BVAIGDIP8 detected by ch341a show only 2M

PS. i got help on #coreboot irc

< hell__> suxx_: the KGPE-D16 has a socketed DIP-8 flash chip
< hell__> you can use a flat screwdriver to *carefully* lift the chip out 

This is spi chip where located it. maybe someone else will come in handy, will be helpful

Windond 25Q16BVAIG datasheet

There is no problems with CH341a programmer

suxx@lab:~/roms$ sudo flashrom -p ch341a_spi -r bios2.bin
flashrom v1.2 on Linux 5.4.0-28-generic (x86_64)
flashrom is free software, get the source code at https://flashrom.org

Using clock_gettime for delay loops (clk_id: 1, resolution: 1ns).
Found Winbond flash chip "W25Q16.V" (2048 kB, SPI) on ch341a_spi.
Reading flash... done.

Libreboot ,for example, works bad. Sometime boot, sometimes not. 9/10 powerups - it shows me black screen. I hope Heads will work better...

[tlaurion]

@tlaurion this is sad :( Maybe set jumper to external card. One Nvidia be for dom0 and Heads. Second for GPU passthrouth to HVM ?

Then in your use case, you would want a functional kgpe-d16_workstation on Aspeed. And make sure you select a GPU that can be passed through (no experience in that).

So the goal here would still be to have gui-init on FBwhiptail for Aspeed onboard configuration.

So you see where to go to make this go forward following your need? @Tonux599 any advancement on https://github.com/osresearch/heads/issues/368#issuecomment-592982240 ?

[ghost]

@tlaurion Are you sure? How to does proxmox works with blacklist nvidia and gpu passthrouth but on stock rom and disabled jumper..or i missing something..

[tlaurion]

@tlaurion do you try to build aspeed VGA driver in dom0 ? Or it doesnt solve issue ?

1- Have Aspeed supported in Heads with FbWhiptail so framebuffer is supported to not have corruption when QubesOS dom0 resuses the FB. Build is happening here. If that doesn't work, it would mean upgrading Heads linux kernel

[tlaurion]

@tlaurion Are you sure? How to does proxmox works with blacklist nvidia and gpu passthrouth but on stock rom and disabled jumper..or i missing something..

https://www.reddit.com/r/Qubes/comments/66wk4q/gpu_passthrough/

[tlaurion]

@tlaurion do you try to build aspeed VGA driver in dom0 ? Or it doesnt solve issue ?

2- Once we have Aspeed FbWhiptail under Heads booting a failing installer, we will be able to open a ticket at QubesOS to have the driver put there if not available.

[ghost]

@tlaurion i need to more read about it. Can you recommend me what to do next? I want to use all the equipment

[tlaurion]

@tlaurion i need to more read about it. Can you recommend me what to do next? Network Server is good but not quite what i wanted. and can we boot OpenBSD with Heads for example?

Your use case should work, but needs people https://github.com/osresearch/heads/issues/692 to tackle with the issues you have to make ASpeed properly supported in framebuffer mode to have a proper workstation.

@Tonux599 was ~interested by a similar use case.

On my part, I'm trying to put together all the pieces that I made in prior so that the ball can be picked up by community members who share common interests while #719 goes forward.

[ghost]

Thanks you @tlaurion . I will follow the news

[tlaurion]

Required work @Tonux599 @pkubaj @0rb677 (following https://github.com/osresearch/heads/issues/692#issue-577966678)

• [ ] Have Aspeed supported in Heads with FbWhiptail so framebuffer is supported to not have corruption when QubesOS dom0 resuses the FB. Build is happening here. If that doesn't work, it would mean upgrading Heads linux kernel
• [ ] Once we have Aspeed FbWhiptail under Heads booting a probable QubesOS failing installer (going in text mode only), we will be able to open a ticket at QubesOS to have the driver put there if not available.

Current PoC

[ghost]

@tlaurion yes you are wizard. it works with FBWHIPTAIL. gui-init works. Still laggy but works. TPM in config need to be disabled if there is no any tpm module on the MB.

Else (usb-init dropped to shell)

TPM_OpenClientCharDev: Could not open char device /dev/tpm0: No such file or directory
Error I/O error from TPM_Extend
!!!! Starting recovery shell 

I use this

#FBWhiptail
CONFIG_CAIRO=y
CONFIG_FBWHIPTAIL=y
export CONFIG_WARNING_BG_COLOR="--background-gradient 0 0 0 150 125 0"
export CONFIG_ERROR_BG_COLOR="--background-gradient 0 0 0 150 0 0"

Similar to x220

Screen

Re-flashing without TPM enabled

[tlaurion]

https://app.circleci.com/pipelines/github/tlaurion/heads/217/workflows/217d72ea-c82a-416f-96e1-7265583a206c/jobs/237

[ghost]

@tlaurion need to fix usb-init while no TPM ? or how to boot Qubes and check if graphical-installer works from shell use kexec?

#!/bin/sh
# Boot a USB installation

. /etc/functions
. /tmp/config

if [ "$CONFIG_TPM" = "y" ]; then
    # Extend PCR4 as soon as possible
    tpm extend -ix 4 -ic usb
fi

usb-scan
recovery "Something failed during USB boot"

[ghost]

@tlaurion added to config

export CONFIG_BOOT_GUI_MENU_NAME="ASUS KGPE-D16 Heads Boot Menu"
export CONFIG_WARNING_BG_COLOR="--background-gradient 0 0 0 150 125 0"
export CONFIG_ERROR_BG_COLOR="--background-gradient 0 0 0 150 0 0"

[tlaurion]

@tlaurion added to config

export CONFIG_BOOT_GUI_MENU_NAME="ASUS KGPE-D16 Heads Boot Menu"
export CONFIG_WARNING_BG_COLOR="--background-gradient 0 0 0 150 125 0"
export CONFIG_ERROR_BG_COLOR="--background-gradient 0 0 0 150 0 0"

@tlaurion need to fix usb-init while no TPM ? or how to boot Qubes and check if graphical-installer works from shell use kexec?

#!/bin/sh
# Boot a USB installation

. /etc/functions
. /tmp/config

if [ "$CONFIG_TPM" = "y" ]; then
  # Extend PCR4 as soon as possible
  tpm extend -ix 4 -ic usb
fi

usb-scan
recovery "Something failed during USB boot"

https://github.com/osresearch/heads/pull/472/commits/c5c98afd177962a381ae9efdd2c3fa9795ae65ad

[tlaurion]

@tlaurion added to config

export CONFIG_BOOT_GUI_MENU_NAME="ASUS KGPE-D16 Heads Boot Menu"
export CONFIG_WARNING_BG_COLOR="--background-gradient 0 0 0 150 125 0"
export CONFIG_ERROR_BG_COLOR="--background-gradient 0 0 0 150 0 0"

https://github.com/osresearch/heads/pull/472/commits/0c7a49c58cd5940d01be1b093f7a5fbf8cd1e700

[ghost]

Cool. Rebuilding.

[tlaurion]

@0rb677 any other bug?

[ghost]

@tlaurion it works stable.

Here is new menu.

Qubes boot from usb Nitrokey

With video cards

fans make noise like crazy. Is it possible to control it?

[ghost]

@tlaurion can we port ASPEED drivers to Qubes and check fan speed?

[ghost]

@tlaurion it cant access to Nitrokey usb card. Cant generate gpg key from Nitrokey. Cant add any gpg keys and reflash bios. Also cant boot freshly installed system. Use ubuntu 20.04 for tests. Change boot device in options do nothing. Too fast happened - hard to debug. I only can boot livecd and install it on hdd. Ubuntu livecd detected Nitrokey attached to usb, but Heads cant

BUS 003 Device 004: ID 20a0:4211 Clay Logic Nitrokey Start

what happened in heads

mount: mounting /dev/sda1 on /boot failed : No such file or directory
!!!!! Unable to mount /boot
!!!!! Starting recovery shell

Random linux distro partition

sda
 |---sda1
 |---sda2
 |---sda3

/dev/sdb1 - usb flash drive

Guixsd installer - boot good but black screen when graphical setup must be IOMMU found.

[tlaurion]

what happened in heads

mount: mounting /dev/sda1 on /boot failed : No such file or directory !!!!! Unable to mount /boot !!!!! Starting recovery shell

Random linux distro partition

sda |---sda1 |---sda2 |---sda3

@0rb677 : seems like you have the PIKE2 controller whichis probably not supported in ``` config/linux-kgpe-d16_*.

EDIT: You can

cp config/linux-kgpe-d16_blah build/linux-blah/.config 
cd build/linux-blah 
make menuconfig 
make savedefconfig 
mv defconfig ../../config/linux-kgpe-d16_blah 

PRs welcome.

it cant access to Nitrokey usb card. Cant generate gpg key from Nitrokey. Cant add any gpg keys and reflash bios. Also cant boot freshly installed system. Use ubuntu 20.04 for tests. Change boot device in options do nothing. Too fast happened - hard to debug. I only can boot livecd and install it on hdd. Ubuntu livecd detected Nitrokey attached to usb, but Heads cant

EDIT: when you try to access the gpg features through menu, enable_usb is called and should find the usb controller from past commit. This is usb1 controller if I recall well which is supposed to be supported with COMPANION_CONTROLLER, and should work since you booted from USB before. What do you get as error? Never played with a nitrokey start, but if usb-init is called from recovery followed by gpg --card-edit or gpg --card-status you should get something. Screen capture would help.

I will put kgpe-d16 builds online, this will facilitate debugging since I have no idea which commit you are trying up. Next comment will be build for which artifacts can be flashed.

[tlaurion]

@0rb677 Actually I won't merge until functional. You can access artifacts from the pull request: https://github.com/osresearch/heads/pull/472

You will see next the commit a green check. Click on it. You will be presented a link to CircleCI. Clikc on it. Click artifacts. You will see https://247-103208611-gh.circle-artifacts.com/0/build/kgpe-d16_workstation/coreboot.rom for commit id https://github.com/osresearch/heads/pull/472/commits/46b9c7948ab476069b3e670c63638355b45943bb

Please report your findings here. You are more then welcome to propose pull requests for working configurations for PIKE controller in linux config. And any other PR that fixes any issue you are encountering.

[tlaurion]

@tlaurion can we port ASPEED drivers to Qubes

That would need a bug report on their side. Start and tag me in.

and check fan speed?

Fan Control

Coreboot does not do fan control so here are your options:

OpenBMC is the best choice for this as you will have fancontrol no matter what the main operating system is doing

Install the OpenBMC port beta to the ASMB4-iKVM or ASMB5-iKVM modules that come with the main KGPE-D16 retail SKU, this provides fan control and a variety of other cool remote management features. The default configuration is 3 pin case fans and 4 pin PWM fans for the CPU fans as this is the only way to provide separate fan control zones due to ASUS not wiring up the rest of the SuperIO fan channels.

Fancontrol/pwmconfig to control your fans via linux.

[SRC](https://www.coreboot.org/Board:asus/kgpe-d16)

You're a question machine!

[tlaurion]

@0rb677 Actually I won't merge until functional. You can access artifacts from the pull request:

472

You will see next the commit a green check. Click on it. You will be presented a link to CircleCI. Clikc on it. Click artifacts. You will see https://247-103208611-gh.circle-artifacts.com/0/build/kgpe-d16_workstation/coreboot.rom for commit id 46b9c79

Please report your findings here. You are more then welcome to propose pull requests for working configurations for PIKE controller in linux config. And any other PR that fixes any issue you are encountering.

I removed ASMBv4 ikvm chip, removed my TPM v2 chip and booted this rom with screen corruption when FB is fired up. What is your experience?

[ghost]

@tlaurion gpg-card issue fixed. dont know why. i use one gpg-card on two devices. Nitrokey Start works good with x220 and KGPE-D16 board.

[tlaurion]

@0rb677 i'm still not sure about your USB keyboard request for workstation on disabled LIBREM_KEY boards (for remote attestation through HOTP). Why not use PS2 keyboard?

Else it would be: KGPE-D16_workstation-usb (no TPM, no HOTP) KGPE-D16_workstation-hotp-verification (once TPM 1.2 verified) KGPE-D16_server (no tpm) KGPE-d16_serve-hotp-verification (Once TPM 1.2 verified)

EDIT: The kgpe-d16_workstation-usb would depend on this

[Tonux599]

@tlaurion this is sad :( Maybe set jumper to external card. One Nvidia be for dom0 and Heads. Second for GPU passthrouth to HVM ?

Then in your use case, you would want a functional kgpe-d16_workstation on Aspeed. And make sure you select a GPU that can be passed through (no experience in that).

So the goal here would still be to have gui-init on FBwhiptail for Aspeed onboard configuration.

So you see where to go to make this go forward following your need? @Tonux599 any advancement on #368 (comment) ?

My use case is disabling the onboard GPU and using a separate one. I'm maintaining a branch which keeps heads running as close to possible to what you would expect from heads on x220/x230. Ensuring that #740 is fixed is currently the only obstacle in obtaining this.

@tlaurion I'm happy to open a PR with my work, however the flavour is along the lines of kgpe-d16-workstation-tpm-{nouveau,radeon,amdgpu} and I'm not sure if this is what the community desires.

[tlaurion]

@tlaurion this is sad :( Maybe set jumper to external card. One Nvidia be for dom0 and Heads. Second for GPU passthrouth to HVM ?

Then in your use case, you would want a functional kgpe-d16_workstation on Aspeed. And make sure you select a GPU that can be passed through (no experience in that). So the goal here would still be to have gui-init on FBwhiptail for Aspeed onboard configuration. So you see where to go to make this go forward following your need? @Tonux599 any advancement on #368 (comment) ?

My use case is disabling the onboard GPU and using a separate one. I'm maintaining a branch which keeps heads running as close to possible to what you would expect from heads on x220/x230. Ensuring that #740 is fixed is currently the only obstacle in obtaining this.

@tlaurion I'm happy to open a PR with my work, however the flavour is along the lines of kgpe-d16-workstation-tpm-{nouveau,radeon,amdgpu} and I'm not sure if this is what the community desires.

@Tonux599 That is kinda the only way, unless we do a kgpe-d16_workstation-external-gpu

[tlaurion]

What does it mean? When i try to sign a /boot. Because Pike and it cant found /dev/sda1 ?

@0rb677 there is no magic here. If PIKE2 is not supported in Heads's kernel, there wont be support for it.

[ghost]

@tlaurion maybe create a room on telegram ? iam flooding too hard here or no? what do you think?

[ghost]

Remove Pike card until enable it in config. And re-test gpg with onboard sata.

[tlaurion]

channel is through u-root on slack, where I won't be the only one answering your questions. :)

[ghost]

This is a Pike card i use before

[Tonux599]

This is a Pike card i use before

@0rb677 That card works with heads if the kernel is built with CONFIG_SCSI_MPT3SAS=y

[ghost]

Rebuilding.

[tlaurion]

I think it reuses the defaults Admin: 12345678

What is default pin when heads generated oem key? https://www.nitrokey.com/documentation/frequently-asked-questions-faq#what-is-the-default-pinpassword

[ghost]

@tlaurion Nitrokey Issues Gnuk Firmware upgrade

[ghost]

I remembered how to do it. HEADS ---> OPTIONS ---> OEM FACTORY RESET It wiil create OEM key Now it works again.

[ghost]

@tlaurion thanks for safeboot. Never tried it.

[ghost]

build with commit https://github.com/osresearch/heads/pull/472/commits/5c21acd889933e763f05d64133e93bb161567391

mpt2sas port enable success in POST

Connect the SAS hard disk drives to SAS connectors 1–4 (red) when installing a 4-port PIKE RAID card from here asus guide

Changed SATA/SAS ports, changed HDD from 3TB to 1TB.

It shows me this

And then dropped to shell. Now i cant access to gui-init menu

!!!! Unable to mount /boot
!!!! Starting recovery shell

If i choose no mount to /boot. it dropped me to shell too.

Dont know why. I go to sleep.

[ghost]

@tlaurion @Tonux599 any ideas how to fix?

[tlaurion]

@0rb677 I never used Pike, don't have the board. I guess the drive that is attempted to be mounted is /dev/sda while yours is probably different?

output of ls /dev/* from recovery shell?

[tlaurion]

@0rb677 I never used Pike, don't have the board. I guess the drive that is attempted to be mounted is /dev/sda while yours is probably different?

output of ls /dev/* from recovery shell?

Know that the board config defines: export CONFIG_BOOT_DEV="/dev/sda1"

So if device is different from Pike then /dev/sda1, board will need to be adjusted.

[ghost]

Serial Attached SCSI controller: Broadcom / LSI SAS2008 PCI-Express Fusion-MPT SAS-2 [Falcon] (rev 03)

no difference, pike or without. in cant mount /dev/sda1. on stock rom it works. on livecd it /dev/sda too, will retry

[Tonux599]

@0rb677 there should be a led flashing persistently on the card to indicate its functioning correctly. I use an SSD with this card and not encountered any problems.

[ghost]

@Tonux599 Blue/Red slots? Led on PIKE works (green light). Will try with SSD.

[ghost]

@tlaurion @Tonux599

OPTIONS -> CHANGE BOOT SETTINGS -> set to /dev/sda5 OPTIONS -> UNTAMPERED BOOT and it will mount read-only / So i can login in system on HDD.

Same problem with OEM RESET - it detect wrong path drive and thats is why i cant reset gpg card and sign /boot files or use default boot menu.

Maybe it cant boot because default ubuntu setup is efi based?

sda
     |----sda1   /boot/efi
     |----sda2  part
     |----sda5   part /

[tlaurion]

I'm happy to open a PR with my work, however the flavour is along the lines of kgpe-d16-workstation-tpm-{nouveau,radeon,amdgpu} and I'm not sure if this is what the community desires.

@Tonux599 please do!

[tlaurion]

@tlaurion @Tonux599

OPTIONS -> CHANGE BOOT SETTINGS -> set to /dev/sda5 OPTIONS -> UNTAMPERED BOOT and it will mount read-only / So i can login in system on HDD.

Same problem with OEM RESET - it detect wrong path drive and thats is why i cant reset gpg card and sign /boot files or use default boot menu.

Maybe it cant boot because default ubuntu setup is efi based?

sda
     |----sda1   /boot/efi
     |----sda2  part
     |----sda5   part /

https://help.ubuntu.com/community/UEFI @0rb677 Legacy mode is supported by Heads and expects a /dev/sda1 to contain grub config.

If you were successful booting from /dev/sda5, you could "Change boot settings" and "save changes" for a /etc/config.user to be written in ROM and applied as an overlay prior of reinstalling.

[ghost]

@tlaurion does it work with TPM 2.0? There is no TPM 1.2 on ebay/amazon.