search

linuxboot / heads

A minimal Linux that runs as a coreboot or LinuxBoot ROM payload to provide a secure, flexible boot environment for laptops, workstations and servers.
https://osresearch.net/
GNU General Public License v2.0
1.41k stars 185 forks source link

Regression: Some peripherals are not discovered since 4.22.01 -> 24.02.01 Coreboot Version bump (upstream: bogus top down allocator) #1740

Closed Oessel closed 1 month ago

Oessel commented 1 month ago

Please identify some basic details to help process the report

A. Provide Hardware Details

  1. What board are you using? (Choose from the list of boards here) Lenovo Thinkpad T430

  2. Does your computer have a dGPU or is it iGPU-only?

    • [x] dGPU (Distinct GPU other then internal GPU)
    • [ ] iGPU-only (Internal GPU, normally Intel GPU)

  3. Who installed Heads on this computer?

  4. What PGP key is being used?

    • [ ] Librem Key (Nitrokey Pro 2 rebranded)
    • [ ] Nitrokey Pro
    • [ ] Nitrokey Pro 2
    • [x] Nitrokey 3 NFC
    • [ ] Nitrokey 3 NFC Mini
    • [ ] Nitrokey Storage
    • [ ] Nitrokey Storage 2
    • [ ] Yubikey
    • [ ] Other

  5. Are you using the PGP key to provide HOTP verification?

    • [x] Yes
    • [ ] No
    • [ ] I don't know

B. Identify how the board was flashed

  1. Is this problem related to updating heads or flashing it for the first time?

    • [ ] First-time flash
    • [x] Updating heads

  2. If the problem is related to an update, how did you attempt to apply the update?

    • [x] Using the Heads menus
    • [ ] Flashrom via the Recovery Shell
    • [ ] External flashing

  3. How was Heads initially flashed?

    • [x] External flashing
    • [ ] Internal-only / 1vyprep+1vyrain / skulls
    • [ ] Don't know

  4. Was the board flashed with a maximized or non-maximized/legacy rom?

    • [x] Maximized
    • [ ] Non-maximized / legacy
    • [ ] I don't know

  5. If Heads was externally flashed, was IFD unlocked?

    • [x] Yes
    • [ ] No
    • [ ] Don't know

C. Identify the rom related to this bug report

V0.2.0-2254-g27d09d4

  1. Did you download or build the rom at issue in this bug report?

    • [ ] I downloaded it
    • [x] I built it

  2. If you downloaded your rom, where did you get it from?

    • [ ] Heads CircleCi
    • [ ] Purism
    • [ ] Nitrokey
    • [ ] Dasharo DTS (Novacustom)
    • [ ] Somewhere else (please identify)

    Please provide the release number or otherwise identify the rom downloaded

  3. If you built your rom, which repository:branch did you use?

    • [x] Heads:Master
    • [ ] Other (please identify)

  4. What version of coreboot did you use in building? { You can find this information from github commit ID or once flashed, by giving the complete version from Sytem Information under Options --> menu}

  5. In building the rom, where did you get the blobs?

    • [ ] No blobs required
    • [ ] Provided by the company that installed Heads on the device
    • [x] Extracted from a backup rom taken from this device(MAC-Adress in gbe.bin)
    • [ ] Extracted from another backup rom taken from another device (please identify the board model)
    • [x] Extracted from the online bios using the automated tools provided in Heads(me.bin)
    • [ ] I don't know

Please describe the problem

Describe the bug A clear and concise description of what the bug is.

The two USB-Ports close to the Power-Button don't work after heads update. (On two devices, t430-maximized with a clean checkout and t430-hotp-maximized from dirty local repository)

To Reproduce Steps to reproduce the behavior:

  1. Go to '...'
  2. Click on '....'
  3. Scroll down to '....'
  4. See error

Expected behavior A clear and concise description of what you expected to happen.

Screenshots If applicable, add screenshots to help explain your problem.

Additional context Add any other context about the problem here.

tlaurion commented 1 month ago

Per https://github.com/linuxboot/heads/blob/master/BOARD_TESTERS.md

@nestire(t430-legacy, t430-maximized) @Thrilleratplay @alexmaloteaux @lsafd @bwachter(iGPU maximized) @shamen123 @eganonoa(iGPU) @nitrosimon @jans23 @icequbes1 (iGPU) @weyounsix (t430-dgpu): Can someone reproduce?

Master merged recently 24.02.01 coreboot version bump was t430 was reported working per @srgrint at https://github.com/linuxboot/heads/pull/1723#issuecomment-2246273211

Which included non-related changes https://github.com/linuxboot/heads/pull/1723/files#diff-17c0a4050b64b2c80df9b9f451dba6a5e9307f56eb84c3e4696e532fa384b6c4 :

I would need someone reproducing this prior of investing time investigating the issue.

@Oessel, others: from recovery shell, can you please:

Oessel commented 1 month ago

IMG_20240801_135956 IMG_20240801_135845

The ports are working under heads, but after booting qubes they aren't recognized. I also tested commit ID 2243 which is also affected and commit id 2221 is working fine

tlaurion commented 1 month ago

@Oessel to my eyes everything works.

What does "USB ports don't work" mean explicitely? Where, when?

Under Heads, we see here EHCI driver enabling everything but USB1 controllers, then xhci enabling USB1 controller. Hence not sure what "USB ports don't work" because not working is not saying anything to be worked on uless more context is given.

Please change title and give more info.From last reply, this is QubesOS/sys-usb related, where non-QubesOS users won't be affected from what I understand.

Please porive OS logs then from sys-usb. Maybe controllers for some reason have changed names? Check under sys-usb being turned off what PCI controller are affected?

tlaurion commented 1 month ago

The ports are working under heads, but after booting qubes they aren't recognized. I also tested commit ID 2243 which is also affected and commit id 2221 is working fine

Ok, so related to top down resource allocation maybe as of now

tlaurion commented 1 month ago

The ports are working under heads, but after booting qubes they aren't recognized. I also tested commit ID 2243 which is also affected and commit id 2221 is working fine

Ok, so related to top down resource allocation maybe as of now

@Oessel note that of now, this is not considered Heads issue and reassigning pci devices under sys-usb qube settings devices tab will most probably fix this and may need to open issue under qubesos.

@marmarek : won't fix on your side too?

marmarek commented 1 month ago

I guess PCI BDF of USB controllers changed after update? If that's the case, I think the firmware update doc (especially EDK-II -> Heads) should have a step to check/update devices assigned to sys-usb and sys-net.

In R4.3 we will have a bit smarter device identification, so at least there will be a clear message what happened (but hopefully it will deal with such changes automatically, at least in some cases).

tlaurion commented 1 month ago

@Oessel ping

Oessel commented 1 month ago

I have three USB controllers listed in the device tab of sys-usb, but with heads v0.2.0-2221 they all work fine, with heads v0.2.0-2243 and 2254 there are the same three controllers but the webcam, the internal smartcard reader and two usb ports are not recognized anymore, thats why i thought it could be heads-related.

tlaurion commented 1 month ago

Ping @nestire(t430-legacy, t430-maximized) @Thrilleratplay @alexmaloteaux @lsafd @bwachter(iGPU maximized) @shamen123 @eganonoa(iGPU) @nitrosimon @jans23 @icequbes1 (iGPU) @weyounsix (t430-dgpu):

Can someone reproduce and give traces of what happen under OS/dom0, this is coreboot bug here.

tlaurion commented 1 month ago

@Oessel I replicated on x230: second msata m.2 and camera were not discovered: USB controllers IDs didn't change. This is coreboot regression.

1742 fixed the regression on my tests on x230. Please validate.

@JonathonHall-Purism no regression on librems with master?

Need of unit tests begins to feel.

Oessel commented 1 month ago

Regression is fixed on t430 too!

tlaurion commented 1 month ago

This TOP DOWN allocator was introduced here by Nico Huber: https://review.coreboot.org/c/coreboot/+/41957

coreboot/coreboot@526c642

@i-c-o-n: Was that tested across all boards?

No, nothing in coreboot is.

Seems to cause regressions still today so not sure why its defconfig?

Because we prefer fixing bugs over working around them. The allocator code has proven to be sound so far. The problems are all around it (in chipset specific code, or even outside coreboot).

Not all fixes had been merged, though (see below).

@i-c-o-n I can provide logs if needed, let me know what you need

Please test with https://review.coreboot.org/c/coreboot/+/80207 applied. If that doesn't help, cbmem and dmesg logs for a start.

Originally posted by @i-c-o-n in https://github.com/linuxboot/heads/issues/1742#issuecomment-2268502708

Alternative PR to not revert to bottom up allocator is pending and will superseed #1742.

tlaurion commented 1 month ago

Instead of reverting regression as mitigation(#1742), apply fix and push testing of fixes applied upstream (#1743)

@i-c-o-n: works on x230.

tlaurion commented 1 month ago 
user@localhost:~/heads$ find config/coreboot-*| while read config; do mmconf=$(sudo grep CONFIG_HWBASE_DEFAULT_MMCONF $config | awk -F "=" {'print $2'}); bitlimit=$(sudo grep CONFIG_DOMAIN_RESOURCE_32BIT_LIMIT $config | awk -F "=" {'print $2'}); echo; echo $config; if [ "$mmconf" == "$bitlimit" ]; then echo "CONFIG_DOMAIN_RESOURCE_32BIT_LIMIT == CONFIG_DOMAIN_RESOURCE_32BIT_LIMIT"; else echo REVIEW_CONFIG;fi;done | grep -B1 REVIEW_CONFIG
config/coreboot-librem_11.config
REVIEW_CONFIG
--
config/coreboot-librem_l1um_v2.config
REVIEW_CONFIG
--
config/coreboot-nitropad-ns50.config
REVIEW_CONFIG
--
config/coreboot-nitropad-nv41.config
REVIEW_CONFIG
--
config/coreboot-p8z77-m_pro-tpm1.config
REVIEW_CONFIG
--
config/coreboot-qemu-tpm1.config
REVIEW_CONFIG
--
config/coreboot-qemu-tpm2.config
REVIEW_CONFIG
--
config/coreboot-t420.config
REVIEW_CONFIG
--
config/coreboot-t430-legacy.config
REVIEW_CONFIG
--
config/coreboot-t430-legacy-flash.config
REVIEW_CONFIG
--
config/coreboot-t520-maximized.config
REVIEW_CONFIG
--
config/coreboot-t530-dgpu-maximized.config
REVIEW_CONFIG
--
config/coreboot-w530-dgpu-K1000m-maximized.config
REVIEW_CONFIG
--
config/coreboot-w530-dgpu-K2000m-maximized.config
REVIEW_CONFIG
--
config/coreboot-x220.config
REVIEW_CONFIG

Analysis:

To be edited after another round of review

tlaurion commented 1 month ago

@i-c-o-n: q35/nv41/ns50 might miss a similar same patch?

user@localhost:~/heads$ sudo grep -Rn -e CONFIG_DOMAIN_RESOURCE_32BIT_LIMIT -e  CONFIG_HWBASE_DEFAULT_MMCONF -e TOP_DOWN config/coreboot-qemu-tpm*.config 
config/coreboot-qemu-tpm1.config:213:CONFIG_DOMAIN_RESOURCE_32BIT_LIMIT=0xfe000000
config/coreboot-qemu-tpm1.config:352:CONFIG_RESOURCE_ALLOCATION_TOP_DOWN=y
config/coreboot-qemu-tpm2.config:210:CONFIG_DOMAIN_RESOURCE_32BIT_LIMIT=0xfe000000
config/coreboot-qemu-tpm2.config:346:CONFIG_RESOURCE_ALLOCATION_TOP_DOWN=y

note: @macpijan nv41/ns50 have TOP_DOWN ress allocator dsabled on heads master.

JonathonHall-Purism commented 1 month ago

@JonathonHall-Purism no regression on librems with master?

Nope, no problems on Librems in master.