Rebootloop on X1C6

[benbender]

Hey, I'm unsure if this is a bug or expected. But anyway, the script seems to be succeeding but the machine won't turn on with the patched bios.

Output of the script as follows:

❯ ./fix_vendor_hashes.py x1c6-bios-patched.bin
Downloading https://github.com/LongSoft/UEFITool/releases/download/A57/UEFIExtract_NE_A57_mac.zip
Extracting UEFIExtract.zip ...
Downloading https://github.com/LongSoft/UEFITool/releases/download/0.28.0/UEFIReplace_0.28.0_mac.zip
Extracting UEFIReplace.zip ...
Generating structure report of x1c6-bios-patched.bin
parseCpdRegion: CPD partition is located inside another CPD partition, skipped
parseCpdRegion: CPD partition intersects with previous one, skipped
parseCpdRegion: CPD partition is located inside another CPD partition, skipped
parseCpdRegion: CPD partition intersects with previous one, skipped
parseRawArea: one of volumes inside overlaps the end of data
parseVendorHashFile: Phoenix hash file found
parsePadFileBody: non-UEFI data found in pad-file
findFitRecursive: FIT table candidate found, but not referenced from the last VTF
findFitRecursive: real FIT table found at physical address FFE90000h
checkProtectedRanges: Phoenix protected range [970000h:DA0000h] hash mismatch, opened image may refuse to boot
parseFptRegion: FPT partition intersects with previous one, skipped
parseFptRegion: FPT partition is located inside another FPT partition, skipped
---------------------------------------------------------------------------
     Address      |   Size    |  Ver  | CS  |          Type / Info
---------------------------------------------------------------------------
_FIT_             | 000000C0h | 0100h | 8Eh | FIT Header |
00000000FFDF0060h | 00019800h | 0100h | 00h | Microcode | CpuSignature: 000406E3h, Revision: 000000DCh, Date: 27.04.2020
00000000FFE09860h | 00017400h | 0100h | 00h | Microcode | CpuSignature: 000406E8h, Revision: 00000026h, Date: 14.04.2016
00000000FFE20C60h | 00019400h | 0100h | 00h | Microcode | CpuSignature: 000806E9h, Revision: 000000D6h, Date: 27.04.2020
00000000FFE3A060h | 00019400h | 0100h | 00h | Microcode | CpuSignature: 000806EAh, Revision: 000000D6h, Date: 27.04.2020
00000000FFF88000h | 00008000h | 0100h | 00h | BIOS ACM | LocalOffset: 00000018h, EntryPoint: 00003BB1h, ACM SVN: 0002h, Date: 28.08.2016
00000000FFFFB440h | 000004BCh | 0100h | 00h | BIOS Init |
00000000FFF90000h | 00005000h | 0100h | 00h | BIOS Init |
00000000FFEA0000h | 0000E800h | 0100h | 00h | BIOS Init |
00000000FFFE0000h | 00001AA4h | 0100h | 00h | BIOS Init |
00000000FFFFAA40h | 00000241h | 0100h | 00h | BootGuard Key Manifest | LocalOffset: 00000000h, KM Version: 10h, KM SVN: 00h, KM ID: 01h
00000000FFFFAF40h | 000002DFh | 0100h | 00h | BootGuard Boot Policy | LocalOffset: 00000500h, BP SVN: 00h, ACM SVN: 02h
Calculating sha256 hashes of FFS volumes ...
Vendor hash table with 4 entries:
 FFS volume @   A0000 (real offset:  8A0000h) hash256 OK
 FFS volume @  170000 (real offset:  970000h) hash256 mismatch! Overriding...
 FFS volume @  5A0000 (real offset:  DA0000h) hash256 OK
 FFS volume @  5E0000 (real offset:  DE0000h) hash256 OK
WARNING: Adding 164 zero padding bytes to the hash table to preserve the original table size
Replacing vendor hash module
parseVolume: unknown file system FFF12B8D-7696-4C8B-A985-2747075B4F50
parseBios: one of volumes inside overlaps the end of data
parseFile: non-empty pad-file contents will be destroyed after volume modifications
File replaced
Fixed ROM image written as x1c6-bios-patched.bin.patched

If I can provide additional infos/files/whatever I'm happy to help...

Thanks anyway!

[sibradzic]

Hi there,

Looks like X1C6 BIOS may have additional security checks in place, such as BootGuard. To check what is the security in place open the BIOS image with UEFITool NE and check the security tab. The purpose of this script is to fix "Phoenix hash file" security check only, and according to your log, this was done correctly (you can run the script again against the x1c6-bios-patched.bin.patched, and it should tell you if all FSS volume check-sums are OK now).

[sibradzic]

Have you tried https://github.com/thrimbor/thinkpad-uefi-sign ?

[benbender]

Hey, thanks for your help!

Background: The problem I've originally tried to solve is a non-working TPM after patching the bios of a Thinkpad X1C6 to get access to the advanced menu of the bios. The patches are working but render the TPM in "MFG"-mode.

Without your patches the system boots fine. With them applied, the system reboots after about 1 second endlessly. It's not the typical "5-beep-you-did-bad"-error or similar, but a straight reboot loop.

Looking at those two images with UEFI-Tool reveals something. The UEFiTool-parser states:

So if I got it correctly, that's what your script intends to fix, right? After applying your script, the parser states the following:

My guess would be that the script overwrites/pads too much data?

I'm not really firm with that kind of stuff and any help is really appreciated. If you need some infos/dumps/etc, I'm glad to provide them. Thanks again in advance!

[sibradzic]

@benbender thanks for the info, this provides some insight. The script is not re-writing any padding on its own, it simply uses UEFIReplace binary to do the replacing work. Your output suggests that UEFIReplace may be wiping out "keyManifest" from modified image. Try removing -all or -asis or both options from: https://github.com/sibradzic/UEFI-playground/blob/6bce52986b1003bfae4c776fe32f2c436da5b5fb/fix_vendor_hashes.py#L247 Alternatively, try manually extracting "keyManifest" from padding and re-adding it after the image has been modified.

[benbender]

I'll try and report back!

[sibradzic]

Someone shared T480 ROM images before and after patching, and looking at them with hex editor confirmed that UEFIReplace wipes out __KEYM__, __ACBP__, __IBBS__ and __PMSG__ entries found at the non-empty padding found near the end of the image. These are BootGuard "Initial Boot Block elements", obviously BootGuard enabled machines won't boot without these, and it is possible that re-injecting same binary data from the non-patched image at same location might fix the issue.

Then there is also the matter of "IBB digest" (found 0x64 bytes following the __IBBS__ header), which will not match the real state of the image following UEFIReplace's intervention, but that might be fixable with some extra work.

Person who shared the T480 image claims that removing -all or -asis does not fix anything.

[benbender]

Thanks for the info!

I just hooked up my X1C6 to my T480. If you want to do some work on this, it would be much appreciated! If I can provide images and/or testing, I'm glad to...

Otherwise I would also able to take some hints and try it myself (without hints I'm feeling a bit lost tbh...)

[sibradzic]

thanks @benbender

If you would like to do some tests on T480, let's do so. I assume you got an external flasher and know how to restore your ROM to original state when things go hairy.

I'll need an original ROM image and the one you patched with something simple (working after re-flash and patched in preferably simple and reproducible way). I can the then subject these images to some hex torture to try to fix TPM or other stuff and you can test them out on your T480. I hope we both learn in the process and that I can improve my scripts with any new insight.

[benbender]

If you would like to do some tests on T480, let's do so. I assume you got an external flasher and know how to restore your ROM to original state when things go hairy.

Sure 👍 And yes, I know how to help myself (mostly ^^). Having access to a T480, X1C6 and a CH341A :)

I'll need an original ROM image and the one you patched with something simple (working after re-flash and patched in preferably simple and reproducible way). I can the then subject these images to some hex torture to try to fix TPM or other stuff and you can test them out on your T480. I hope we both learn in the process and that I can improve my scripts with any new insight.

Applied all of those patches: https://github.com/digmorepaka/thinkpad-firmware-patches/blob/master/xx70_xx80_patches_v6.txt via UEFIPatch. Could reduce to those for the advanced menu if needed. If those patches are simple enough, I could instantly provide both images.

We could also do it live via Gitter. You can find me on https://gitter.im/YogaSMC/community.

[sibradzic]

OK, just pass me the original image, and than the one with advanced menu only. I assume you can boot with advanced menu patch, but TPM is broken?

[benbender]

Did not try without the other patches yet, but I assume yes.

And yes, I can boot with those patches and broken TPM with the following, additional bin-patch to disable "Phoenix BIOSGUARD":

# works with paranoidbashthot's bypass method,
's/\x4C\x4E\x56\x42\x42\x53\x45\x43\xFB\xFF/\x4C\x4E\x56\x42\x42\x53\x45\x43\xFF\xFF/g'

Will try tomorrow and provide the images.

Thanks again!

[benbender]

Hey! Sorry for being late at coming back to your generous offer of help! Didn't found the time yesterday, but here we go! Thanks again!

I attached all relevant files and a log including outputs of what I've done. Please note the UEFIPatch-errors!

Read original, resetted bios. Verified working including TPM.

bb@bbs-MacBook-Pro t480-biosmod % flashrom -p ch341a_spi -r t480-original-bios-v1.34-resetted.rom
flashrom v1.2 on Darwin 20.1.0 (x86_64)
flashrom is free software, get the source code at https://flashrom.org

Calibrating delay loop... OK.
libusb: info [darwin_claim_interface] no interface found; setting configuration: 1
Found Winbond flash chip "W25Q128.V" (16384 kB, SPI) on ch341a_spi.
Reading flash... done.

Patch bios with "advanced menu"-patches.

Note the errors by UEFIPatch (which I didn't noticed before)

bb@bbs-MacBook-Pro t480-biosmod % ./UEFIPatch ./t480-original-bios-v1.34-resetted.rom ./xx70_xx80_patches_v6.txt -o ./t480-patched-bios-v1.34-adv-menu.rom
parseVolume: unknown file system FFF12B8D-7696-4C8B-A985-2747075B4F50
parseBios: one of volumes inside overlaps the end of data
parseFile: non-empty pad-file contents will be destroyed after volume modifications
patch: replaced 16 bytes at offset 3B60h 04320B483CC2E14ABB16A73FADDA475F -> 778B1D826D24964E8E103467D56AB1BA
patch: replaced 16 bytes at offset 118D0h 04320B483CC2E14ABB16A73FADDA475F -> 778B1D826D24964E8E103467D56AB1BA
Image patched

Flash back the patched bios without the bin-replace for testing. Can't boot. Thinkpad does only the "5-beep-you-did-bad-error".

bb@bbs-MacBook-Pro t480-biosmod % flashrom -p ch341a_spi -w t480-patched-bios-v1.34-adv-menu.rom
flashrom v1.2 on Darwin 20.1.0 (x86_64)
flashrom is free software, get the source code at https://flashrom.org

Calibrating delay loop... OK.
Found Winbond flash chip "W25Q128.V" (16384 kB, SPI) on ch341a_spi.
Reading old flash chip contents... done.
Erasing and writing flash chip... Erase/write done.
Verifying flash... VERIFIED.

Applied the bin replace and flashed. Everything is working as expected. Adv menu shows up, iGPU-freqs & LCD-control missing. No beep-errors, TPM in MFG-mode

bb@bbs-MacBook-Pro t480-biosmod % flashrom -p ch341a_spi -w t480-patched-bios-v1.34-adv-menu-bin-patch.rom
flashrom v1.2 on Darwin 20.1.0 (x86_64)
flashrom is free software, get the source code at https://flashrom.org

Calibrating delay loop... OK.
libusb: info [darwin_claim_interface] no interface found; setting configuration: 1
Found Winbond flash chip "W25Q128.V" (16384 kB, SPI) on ch341a_spi.
Reading old flash chip contents... done.
Erasing and writing flash chip... Erase/write done.
Verifying flash... VERIFIED.

t480-original-bios-v1.34-resetted.rom.zip t480-patched-bios-v1.34-adv-menu-bin-patch.rom.zip t480-patched-bios-v1.34-adv-menu.rom.zip xx70_xx80_patches_v6.txt.zip

[sibradzic]

@benbender

OK, I've played a bit with your 480-patched-bios-v1.34-adv-menu.rom.zip... I've applied fix_vendor_hashes.py and re-injected BootGuard __KEYM__, __ACBP__, __IBBS__ and __PMSG__ data to the image. There are two manually modified ROM images in the this tar.xz archive:

1. t480-patched-bios-v1.34-adv-menu.rom.patched.cloned_ibb (md5: 98998f9b6f290174c161e8b2f51f91ea)
2. t480-patched-bios-v1.34-adv-menu.rom.patched.cloned_ibb_fixed_digest (md5: 036ee8964f60f4a9e64029cd0a905348)

1st one is your original adv-menu + fix_vendor_hashes + manually injected BootGuard data. 2nd one is the same + manually repaired BootGuard protected range digest. You can try flashing 1st and 2nd one, and please let me know if they worked.

Also, if none of them work, try applying 's/\x4C\x4E\x56\x42\x42\x53\x45\x43\xFB\xFF/\x4C\x4E\x56\x42\x42\x53\x45\x43\xFF\xFF/g' on over the images and check if TPM is still in MFG-mode.

Thanks in advance!

[benbender]

@sibradzic Uh, very nice. Will test in a few hours and report back. Thank YOU! :)

[benbender]

@sibradzic I'm sorry to report that both images produced the exact same result as described in the first post.

I flashed both images as normal, tried to turn on the machine, the power led lid up for a second and the machine instantly powered off after that... So no success :(

Additional thought: The initial corruption seems to take place within the initial adv-menu-patch. May it be the case that, with fixing the hashes, this initial error comes to effect and leads to the crash now (while being ignored before because of the faulty hash)?

[sibradzic]

Have you tried applying 4C4E564242534543FBFF -> 4C4E564242534543FFFF hack on top of the two images I've sent?

[benbender]

Nope, I didn't. If that would have been the problem, the system would have reacted with a 5-tone-beep-error on boot, but it simply powered off... Do you think that would have made a difference? If yes, I could try it, but my feeling says that it would be the right move, to fix the initial UEFIPatch-Error while applying the advmenu-patch. Do you see any chance for that?

[sibradzic]

I think you should definitely try it out, as it may determine further steps in effort to sort this out...

[benbender]

@sibradzic I'll expect the same result (poweroff) but will try it happily later today. Shall I try both image-variants with bin-patch?

[sibradzic]

yes please.

[benbender]

@sibradzic Sorry for being late again, but I tried it just yet and I'm sorry to report that the result is as I've had expected it to be. Both bin-patched versions lead to the same "instant power off"-behaviour as before...

[sibradzic]

That's a bummer. Well, thanks for testing anyway! Let me think for a while what could be the next step...

[benbender]

@sibradzic My take would be to try to fix the patches in the first place. Or am I on the wrong track thinking that this is the root of the problem?

Patch bios with "advanced menu"-patches.

Note the errors by UEFIPatch (which I didn't noticed before)

bb@bbs-MacBook-Pro t480-biosmod % ./UEFIPatch ./t480-original-bios-v1.34-resetted.rom ./xx70_xx80_patches_v6.txt -o ./t480-patched-bios-v1.34-adv-menu.rom
parseVolume: unknown file system FFF12B8D-7696-4C8B-A985-2747075B4F50
parseBios: one of volumes inside overlaps the end of data
parseFile: non-empty pad-file contents will be destroyed after volume modifications
patch: replaced 16 bytes at offset 3B60h 04320B483CC2E14ABB16A73FADDA475F -> 778B1D826D24964E8E103467D56AB1BA
patch: replaced 16 bytes at offset 118D0h 04320B483CC2E14ABB16A73FADDA475F -> 778B1D826D24964E8E103467D56AB1BA
Image patched

[benbender]

@sibradzic Just came across this really interesting read and thought I should share it with you as it, while basically unrelated, reveals probably why the machine is rebooting after patching: https://trmm.net/Sleep_attack/

Relevant quote:

If any of the KEYM or ACBP signature checks fail, or if the IBB hash comparison fails, then the ACM takes action depending on the Bootguard OTP configuration. These Bootguard configurations available have not publicly disclosed by Intel, although the FIT tool has several profiles that can be selected related to the security configuration and some researchers have tried to reverse engineer it: [...] Many OEMs seem to select the "immediate shutdown" option, although there some choose "do nothing" and push validation elsewhere, which can lead to its own set of vulnerabilities.

So I would deduce that the signature checks are, at least in some way, intact on boot with the patched firmware with broken TPM and are not after our patching attempts. Unsure if it gives you a new vector of thinking, but as the article / bug is interesting anyway, I thought it wouldn't hurt ;)

[benbender]

Hey,

did some testing yesterday (see https://github.com/tylernguyen/x1c6-hackintosh/issues/85#issuecomment-730309699) on my X1C6 which has the same behaviour than the T480. From what I see those errors mentioned above are appearing with UEFIPatch/UEFIReplace regardless of which patch is applied. It may be a problem/bug in those tools and not on your end. The workaround atm is to patch the TPM into MFG Mode using the binpatch. I've opened an issue at the
@UEFITool-Repo: https://github.com/LongSoft/UEFITool/issues/227.

If I didn't got something wrong my impression is that we have to wait for a reaction as everything else seems to be a dead end to me.

[sibradzic]

You are probably right, it is likely that UEFITools tools render the image unbootable, and there is little we can do. The main issue here is that both UEFIPatch & UEFIReplace are tools found @ "old" UEFITool branch (as opposed to "NE" branch), meaning the parser logic in these older tools may fail to understand & re-build more recent UEFI structures. I wonder would an effort to port patch & replace tools to "NE" be reasonable...

[benbender]

Does seem to be a known "issue": https://github.com/LongSoft/UEFITool/issues/172