search

corna / me_cleaner

Tool for partial deblobbing of Intel ME/TXE firmware images
GNU General Public License v3.0
4.51k stars 279 forks source link

Adding soft-disable support for IFWI/ME12 #282

Open dt-zero opened 5 years ago

dt-zero commented 5 years ago

Based on the work of @davidmartinzeus

IMPORTANT: If you are new here, please read my comment at https://github.com/corna/me_cleaner/pull/282#issuecomment-992988892

dt-zero commented 5 years ago

On Intel Cannonlake H (300) Series chipsets such as QM370, Q370, B360, H370, H310, Z390, C242, C246, HM370, CM246 and QMS380 with Intel ME 12, the High Assurance Platform bit has been moved to PCHSTRP32.

My testing board is an ASROCK H370M-ITX/ac. With this patch, I get the usual good signs of ME disablement. ME disappeared from the PCI bus and the OEM bios shows ME version 0.0.0.0.

me_down

This patch is the very definition of bare minimum. A proper implementation would consist of parsing the CSE Partition Layout Table to locate the Boot Partition Descriptor Table and then finding a type 2 entry (FTPR). (Like how @platomav 's wonderful MEAnalyzer does)

Other than that, we would probably benefit from a cleaner way of tracking the HAP bit and matching that to specific chipsets. I think the best option there involves parsing the MFS and checking the "008"/"Home Directory" partition's "bup/bup_sku/plat_n_sku", which holds an SKU identifier inside the BIOS image. My current understanding is that any SKU identifier corresponds to a set PCH strap layout and hence a HAP bit position in the rom.

craigacgomez commented 5 years ago

I can confirm that this works on Clevo/Sager P95XER series notebooks (Tested with a P955ER) with ME12.

Original BIOS backup (using Intel FPT)

python ./me_cleaner.py -c ORIGINAL_BIOS.bin
Full image detected
Found FPT header at 0x2000
Found 7 partition(s)
Found FTPR header: FTPR partition spans from 0x88000 to 0x88000
Found FTPR manifest at 0x88250
ME/TXE firmware version 12.0.40.1433 (generation 4)
Public key match: Intel ME, firmware versions 12.x.x.x
The HAP bit is NOT SET
Checking the FTPR RSA signature... VALID

Applying me_cleaner with this pull request #282 

python ./me_cleaner.py -s -O PATCHED_BIOS.bin ORIGINAL_BIOS.bin
Full image detected
Found FPT header at 0x2000
Found 7 partition(s)
Found FTPR header: FTPR partition spans from 0x88000 to 0x88000
Found FTPR manifest at 0x88250
ME/TXE firmware version 12.0.40.1433 (generation 4)
Public key match: Intel ME, firmware versions 12.x.x.x
The HAP bit is NOT SET
Setting the HAP bit in PCHSTRP32 to disable Intel ME...
Checking the FTPR RSA signature... VALID
Done! Good luck!

Backup after flashing me_cleaner patched BIOS (patched BIOS flashed using Intel FPT, backup using Intel FPT)

python ./me_cleaner.py -c NEW_BACKUP.bin
Full image detected
Found FPT header at 0x2000
Found 7 partition(s)
Found FTPR header: FTPR partition spans from 0x88000 to 0x88000
Found FTPR manifest at 0x88250
ME/TXE firmware version 12.0.40.1433 (generation 4)
Public key match: Intel ME, firmware versions 12.x.x.x
The HAP bit is SET
Checking the FTPR RSA signature... VALID

Output of Intel CSME MEInfo after flashing me_cleaner patched BIOS

sudo ./MEInfo
Intel (R) MEInfo Version: 12.0.40.1433
Copyright (C) 2005 - 2019, Intel Corporation. All rights reserved.

Error 198: ME disabled.
Espionage724 commented 5 years ago

Per https://github.com/corna/me_cleaner/issues/3#issuecomment-554487267, this PR was required and works fine for me on a Coffee Lake system.

roema commented 5 years ago

Hi!

I have a brand new Lenovo E490 with ME 12.0.1427.35

The bios chip is very easy to access. So i have created a dump with flashrom.

After flashing the patched image. lspci shows:

Intelmetool say on the first device 0x9de0:


MEI found: [8086:9de0] Device 9de0

ME Status   : 0xa0000245
ME Status 2 : 0xf10506

ME: FW Partition Table      : OK
ME: Bringup Loader Failure  : NO
ME: Firmware Init Complete  : YES
ME: Manufacturing Mode      : NO
ME: Boot Options Present    : NO
ME: Update In Progress      : NO
ME: Current Working State   : Normal
ME: Current Operation State : M0 with UMA
ME: Current Operation Mode  : Normal
ME: Error Code              : No Error
ME: Progress Phase          : ROM Phase
ME: Power Management Event  : Clean Moff->Mx wake
ME: Progress Phase State    : (null)

ME: Extend Register not valid

ME: Firmware Version 12.0.1427.35 (code) 12.0.1427.35 (recovery) 12.0.1427.35 (fitc)

ME Capability: Full Network manageability                 : OFF
ME Capability: Regular Network manageability              : OFF
ME Capability: Manageability                              : OFF
ME Capability: Small business technology                  : OFF
ME Capability: Level III manageability                    : OFF
ME Capability: IntelR Anti-Theft (AT)                     : OFF
ME Capability: IntelR Capability Licensing Service (CLS)  : ON
ME Capability: IntelR Power Sharing Technology (MPC)      : OFF
ME Capability: ICC Over Clocking                          : OFF
ME Capability: Protected Audio Video Path (PAVP)          : ON
ME Capability: IPV6                                       : OFF
ME Capability: KVM Remote Control (KVM)                   : OFF
ME Capability: Outbreak Containment Heuristic (OCH)       : OFF
ME Capability: Virtual LAN (VLAN)                         : ON
ME Capability: TLS                                        : OFF
ME Capability: Wireless LAN (WLAN)                        : OFF

Laptop is working normaly, no shutdown after 30 mins. 

I think, ME12 is ignoring the HAP Bit.
dt-zero commented 5 years ago

Hi @roema ! :) I think it is unlikely that any ME versions lack the HAP bit behaviour. I'm suspecting that the Whiskey Lake PCH layout is different. Can you send me your original BIOS dump?

roema commented 5 years ago

Hi @dt-zero !!

Thanks for you message! Yes! Here is the link E490

Thanks for your work!!

dt-zero commented 5 years ago

@roema Okay, I see what happened here :)

This is the Cannonlake LP variant with a different PCH strap layout. Fixing this in the general case will take some more work, but for now, we can work around this. Try this version. I'm glad if you find my contributions useful :tada:

me_cleaner.py

roema commented 5 years ago

@dt-zero you are the best! It seems to be working!

Now the device /dev/eim0 is gone and in UEFI BIOS no ME Version shown. ME

Thank you very much!

beaker23 commented 5 years ago

This works on MSI B360 Gaming plus as well. I used the me_cleaner.py file from this pull request's commit.

The one posted as a text file did not work (it reported success using me_cleaner.py -c, but when flashed did not neutralize the me)

me-clean me-clean2

dt-zero commented 5 years ago

@beaker23 Glad to hear it works for you as well!

For future reference, there is Cannonlake-H series ( QM370, Q370, B360, H370, H310, Z390, C242, C246, HM370, CM246, QMS380) and Cannonlake-LP. I'm not sure which chipsets are in LP, but generally, if you have ME 12 and its not one of the H series I listed, it is probably LP. The script from the commit is for H series, the one I attached is for LP. Making one script that is compatible with both would involve (like the original message mentions) parsing the ME file system to identify the SKU or something similar. I'm not sure to what extent @corna would like to see me_cleaner modified to accommodate this so I tried to make my changes as small as possible. (Which has its ups and downs as you can see)

dtbartle commented 4 years ago

Commit from PR works on Z390M-ITX/AC.

suut commented 4 years ago

Hello @dt-zero, the me_cleaner.py for LP variant of cannonlake you posted is identical to the one from the commit, is there some kind of mix-up there ? The commit variant did nothing for me, I have the LP variant.

On another note since you seem pretty knowledgeable, what can you say about the "Temporarily disable" mode of ME ? It's an hidden option in my BIOS that I can enable and makes ME totally disappear from the PCI bus and no tool can interact with it afterwards , it's doing more than HAP/AltMeDisable on my desktop computer it seems.

dt-zero commented 4 years ago

Hi @suut , there is a difference between the two, its the offset in the PCH strap region. LP is offset 0x70, while Cannonlake-H is 0x80. If you have an LP board, use the python script I attached. (Not the one from the commit)

Functionality wise, the HAP bit does the same. If you successfully enable the HAP bit, the ME disappears from the PCI bus and no tool will be able to interact with it.

suut commented 4 years ago

Hi @suut , there is a difference between the two, its the offset in the PCH strap region. LP is offset 0x70, while Cannonlake-H is 0x80. If you have an LP board, use the python script I attached. (Not the one from the commit)

Functionality wise, the HAP bit does the same. If you successfully enable the HAP bit, the ME disappears from the PCI bus and no tool will be able to interact with it.

I meant the two files were byte-for-byte equal but I guess I mixed up downloads, thank you for your feedback, I will try again now.

On another note, how did you find the offset ? Using Intel FIT, toggling the "Reserved" setting and looking at which offset in the image the bit was toggled ? So like at byte 371 for Cannonlake-LP

dt-zero commented 4 years ago

I meant the two files were byte-for-byte equal but I guess I mixed up downloads, thank you for your feedback, I will try again now.

On another note, how did you find the offset ? Using Intel FIT, toggling the "Reserved" setting and looking at which offset in the image the bit was toggled ? So like at byte 371 for Cannonlake-LP

They are definitely not equal byte-for-byte or otherwise.

My understanding is that Intel (CS)ME System Tools (which include FIT) are not publicly released to end-users but only to OEMs, so naturally I would not try obtain them in any way.

However, purely theoretically, had someone done that, they could extract the platform descriptor XMLs from FIT and use grep to find a sequence of entries that had the same distribution of bits between fields as with Intel ME 11. That way, they could find out that the old entry that used to be called "reserve_hap" is now "PCH_Strap_CSME_CSE_CSME_Reserved". The same xml node would specify the offset and which bit of that entry controls the HAP mode.

suut commented 4 years ago

I meant the two files were byte-for-byte equal but I guess I mixed up downloads, thank you for your feedback, I will try again now. On another note, how did you find the offset ? Using Intel FIT, toggling the "Reserved" setting and looking at which offset in the image the bit was toggled ? So like at byte 371 for Cannonlake-LP So They are definitely not equal byte-for-byte or otherwise.

My understanding is that Intel (CS)ME System Tools (which include FIT) are not publicly released to end-users but only to OEMs, so naturally I would not try obtain them in any way.

However, purely theoretically, had someone done that, they could extract the platform descriptor XMLs from FIT and use grep to find a sequence of entries that had the same distribution of bits between fields as with Intel ME 11. That way, they could find out that the old entry that used to be called "reserve_hap" is now "PCH_Strap_CSME_CSE_CSME_Reserved". The same xml node would specify the offset and which bit of that entry controls the HAP mode.

Oh I see, thanks for the information !

So for other people interested by some googling we can find cnp_h_rvp.xml and cnp_lp_rvp.xml which contain the PCH straps for Cannonlake H and Cannonlake LP respectively. (I don't know if these documents are too touchy to link in here, feel free to tell me if I need to edit)

pedrib commented 4 years ago

@dt-zero Success! This worked on my Dell 7060 MFF (Intel Q370 / Cannon Lake). Dumped the bios via FPT, cleaned it with your me_cleaner patch using the -s for HAP bit enable, flashed it back and it works. There's no info in the BIOS about ME, but Linux can't find anything about it using Intel and other tools, even with the mei kernel modules loaded. 

sudo ./intelmetool -d -m
ME PCI device is hidden
RCBA addr: 0x00000000
Can't find ME PCI device

There's no ME device on lspci, and me_cleaner -c says:

Full image detected
Found FPT header at 0x4000
Found 7 partition(s)
Found FTPR header: FTPR partition spans from 0x168000 to 0x168000
Found FTPR manifest at 0x168268
ME/TXE firmware version 12.0.40.1433 (generation 4)
Public key match: Intel ME, firmware versions 12.x.x.x
The HAP bit is SET
Checking the FTPR RSA signature... VALID
mtaillefumier commented 4 years ago

I tried deactivating the hap bit on the asus ws z390 pro but no luck. I get an infinite boot loop. I also tried differently recreating the bios using others tools with hap bit on, same result.

privacyguy123 commented 4 years ago

Tried this on my machine - sound doesn't work and it takes over a minute to boot ... is this the intended outcome?

EDIT: Is it possible that ME 12 could be supported by whitelisting some more of the modules?

Intel (R) Firmware Update Utility Version: 12.0.65.1567
Copyright (C) 2005 - 2020, Intel Corporation. All rights reserved.

Checking firmware parameters...

Warning: Do not exit the process or power off the machine before the firmware update process ends.

Error 509: Mandatory partitions (FTPR / NFTP / RBEP) were not found in the Update Image.
dt-zero commented 4 years ago

@MattBJ I fixed it for your version, attached below.

Run it with python me_cleaner.py -s orig.bin -O cleaned.bin

me_cleaner.py.txt

dt-zero commented 4 years ago

@MattBJ the PCH strap configuration is different for all these platforms, that's what I'm changing most of the time, however in your case I also added the ME14 public key to the list, just to clear the associated warning. Let me know if you achieve the desired results, cheers! :smiley:

dt-zero commented 4 years ago

@MattBJ

So fingers crossed this works! I really appreciate your work, and everybody else's here.

Awesome, good luck!

If I wanted to neutralize and shrink the ME section of the ROM, would I just do this:

Changing the layout of the rom, such as shrinking or deleting partitions is not supported since the IFWI format (starting from ME12) was introduced. The original script here only allowed you to do that, because it didn't recognize ME 14, since I only hardcoded that check for ME12. (Which was the latest version when I wrote that code) So no shrinking or deleting partitions, unless you add support yourself, sorry :frowning_face:

Also, if you happened to know, what would be the general purpose of shrinking this portion?

As far as I'm aware, the purpose of shrinking, would probably be to gain more usable space on the BIOS chip, which would come handy in case of using a custom BIOS like Coreboot. I wouldn't worry about that with the OEM bios.

And, also, when getting bios updates, will I always have to scrub out the ME or will that piece be unaffected throughout the life of the motherboard unless I decide to revert to my saved original copies?

BIOS updates will restore the flash's content, so you will have to do this procedure every time you update your BIOS.

Once again thank you for your time!

You're welcome, I'm glad people are showing interest in this, so at least on paper Intel has some incentives to fix their crap.

spicewalker commented 4 years ago

@dt-zero

Confirmed working on a Lenovo Yoga X390 20NN00F8GE with ME 12.0.35.1427 (Cannonlake-LP). Attached a CH341A programmer to the BIOS chip next to the PCIe SSD (a 256MBit MXIC MX25l25673G that flashrom recognized as MX25L25635F/MX25L25645G, though it handled the IC without a hitch). Binary diff shows that me_cleaner set the bit 0 at address 0x173 (from 0x18 to 0x19), otherwise the dumps before and after are equal.

ME Version in BIOS setup is gone, as well as /dev/mei0. Great success! I noticed two things though.

Anyway, I would like to thank you for the time that you invested into me_cleaner for handling ME12. Do you think the recent Intel leaks could advance this project further? Is there even still interest in this? The last commit dates back to 2018...

blemau commented 4 years ago

@MattBJ and @dt-zero are you or anyone else able to confirm whether setting the High Assurance Platform bit and soft-disabling the ME still works on platforms with ME versions above version 12?

Yannik commented 3 years ago

Thanks for your work, @dt-zero I tried using it to remove ME 12 on my nuc8 (i5-8259U, coffee lake).

Unfortunately, ME is still present after flashing the rom with the HAP bit set.

Could you have a look at what is going wrong here?

https://github.com/Yannik/nuc8-firmware/raw/main/orig.rom https://github.com/Yannik/nuc8-firmware/raw/main/hap-bit-set.rom

dt-zero commented 3 years ago

Hi @Yannik

You got the LP series variant, same case as @roema . I attached your rom with the HAP bit set. hap_set.zip

For future reference, you can use the modified script I linked in my previous answer: https://github.com/corna/me_cleaner/pull/282#issuecomment-554627204

dt-zero commented 3 years ago

@MattBJ and @dt-zero are you or anyone else able to confirm whether setting the High Assurance Platform bit and soft-disabling the ME still works on platforms with ME versions above version 12?

Hey @blemau . Yeah I can confirm it still works.

Yannik commented 3 years ago

@dt-zero thank you, this worked!! :)

dt-zero commented 3 years ago

Hey @spicewalker .

I'm glad to hear your success.

  • I did several BIOS updates (1.69 -> 1.72 -> 1.74 -> 1.76) and the HAP bit was not reset in any of them. Apparently the Lenovo BIOS update scenario does not handle it.

Yes, that is very plausible. Some OEM update processes don't touch the PCH descriptor region, it may vary from vendor to vendor.

  • There is a concerning power drain when the machine is off (about 10% per day). This happens only after setting the HAP bit. Not a big concern for me as I mainly use the machine with power sockets in reach, though I worry a little for battery degradation. Maybe it has something to do with the PMIC? Did you guys measure power consumption when your machines are off?

This is also plausible, as the CSME is responsible for some of the power management jobs. It is unfortunately something you'll have to accept if you choose to shut down the ME in this fashion.

Anyway, I would like to thank you for the time that you invested into me_cleaner for handling ME12. Do you think the recent Intel leaks could advance this project further?

You are very welcome, I'm glad to see the continued interest from all of you in this. I don't see any of the recent Intel as useful for this. The main blocker has always been about "owning" the platform in the sense of the signing keys. Unless Intel releases their signing keys, I don't see any way we'll truly own the platform even to the degree of freeing the ME, let alone to have transparent microcode updates.

Is there even still interest in this? The last commit dates back to 2018...

Well, there is interest, about as much as you can see on this thread I would assume. Sometimes people show up, I try to help them and usually everyone is kind and grateful for the help, but I would imagine on the scale of all Intel customers, we are still in the 0.001% of people who understand what this is and care about it to the degree of doing something about it. In case you are curious about more of my thoughts, you can watch one of my previous talks on this.

https://www.youtube.com/watch?v=MqR6i6qvZFc

Kind regards

jhsvip commented 3 years ago

@MattBJ I fixed it for your version, attached below.

Run it with python me_cleaner.py -s orig.bin -O cleaned.bin

me_cleaner.py.txt

Hi @dt-zero ASROCK B360M-HDV did not work,The me version still exist in Advanced-->>Chipset not 0.0.0.0 After dumped the moded bios the script result here: python me_cleaner.py -s dunmped2.bin -O cleaned2.bin Full image detected Found FPT header at 0x4000 Found 8 partition(s) Found FTPR header: FTPR partition spans from 0x8e000 to 0x8e000 Found FTPR manifest at 0x8e250 ME/TXE firmware version 12.0.49.1534 (generation 4) Public key match: Intel ME, firmware versions 12.x.x.x The HAP bit is SET Setting the HAP bit in PCHSTRP32 to disable Intel ME... Checking the FTPR RSA signature... VALID Done! Good luck!

Did i do something messed up?Could u help me?

Yannik commented 3 years ago

@jhsvip I suggest you use intelmetool to check if me is still present or not.

dt-zero commented 3 years ago

@MattBJ I fixed it for your version, attached below. Run it with python me_cleaner.py -s orig.bin -O cleaned.bin me_cleaner.py.txt

Hi @dt-zero ASROCK B360M-HDV did not work,The me version still exist in Advanced-->>Chipset not 0.0.0.0 After dumped the moded bios the script result here: python me_cleaner.py -s dunmped2.bin -O cleaned2.bin Full image detected Found FPT header at 0x4000 Found 8 partition(s) Found FTPR header: FTPR partition spans from 0x8e000 to 0x8e000 Found FTPR manifest at 0x8e250 ME/TXE firmware version 12.0.49.1534 (generation 4) Public key match: Intel ME, firmware versions 12.x.x.x The HAP bit is SET Setting the HAP bit in PCHSTRP32 to disable Intel ME... Checking the FTPR RSA signature... VALID Done! Good luck!

Did i do something messed up?Could u help me?

@jhsvip I suggest you double check what you've done. Since it seems alright to me from here. Your board is an H series cannonlake board, meaning it is indeed PCHSTRP32. Check if offset 0x182 in the "cleaned" image is 0x11. (0x11 = hap bit on, 0x10 = hap bit off)

jhsvip commented 3 years ago

@MattBJ I fixed it for your version, attached below. Run it with python me_cleaner.py -s orig.bin -O cleaned.bin me_cleaner.py.txt

Hi @dt-zero ASROCK B360M-HDV did not work,The me version still exist in Advanced-->>Chipset not 0.0.0.0 After dumped the moded bios the script result here: python me_cleaner.py -s dunmped2.bin -O cleaned2.bin Full image detected Found FPT header at 0x4000 Found 8 partition(s) Found FTPR header: FTPR partition spans from 0x8e000 to 0x8e000 Found FTPR manifest at 0x8e250 ME/TXE firmware version 12.0.49.1534 (generation 4) Public key match: Intel ME, firmware versions 12.x.x.x The HAP bit is SET Setting the HAP bit in PCHSTRP32 to disable Intel ME... Checking the FTPR RSA signature... VALID Done! Good luck! Did i do something messed up?Could u help me?

@jhsvip I suggest you double check what you've done. Since it seems alright to me from here. Your board is an H series cannonlake board, meaning it is indeed PCHSTRP32. Check if offset 0x182 in the "cleaned" image is 0x11. (0x11 = hap bit on, 0x10 = hap bit off)

Hi, I found the problem ,Me_cleaner located the hap bit address is 0x172, the real hap bit is 0x182 as you say.

tinylabs commented 3 years ago

@craigacgomez - Where did you find the Intel FPT tool? Have the same hw.

craigacgomez commented 3 years ago

@tinylabs you can get it from the Intel CSME 12 package (https://www.win-raid.com/t596f39-Intel-Management-Engine-Drivers-Firmware-amp-System-Tools.html)

ghost commented 3 years ago

Hello, Does anyone know how to modify the me_cleaner script to flip the HAP bit again to enable Intel ME? I used softdisable only (-s) on Intel ME v11, but I lost the original dump. Now I need to reenable it again. Thank you!

ElVasquito commented 3 years ago

Hi, just tried on a Dell (don't know its exact chipset, has an i7-6500), dumped the orig FW twice (just in case) applied the original me_cleaner as well as your improved version (both made the same changes), and after burning, the machine will not post anymore. Just a shine of the power LED when the charger is plugged, then it goes off after a second and that's it. Dead. Maybe Dell is doing some special check? Attached is the orig FW, any clues appreciated!

DellFW.zip

ElVasquito commented 3 years ago

Worth mentioning that after reflashing the original, system starts normally :/

melvyn2 commented 3 years ago

Worked on an Intel Core i9-9900k w/ Gigabyte Z390 Aorus Master. Extracted firmware v f11n using flashrom, then reflashed with flashrom -p internal, result is:

 sudo intelmetool -m
Can't find ME PCI device

System still runs after 30 mins.

Giga-Pudding commented 3 years ago

I used flashrom with a ch341a_spi external flasher. It worked perfectly for me, thanks alot!

MSI_SnapShot_00

xnuken commented 3 years ago

I can confirm this works tried it on my ROG MAXIMUS XI GENE 930B7E6F-E709-43D1-88AD-85680322CB2F

DodoidOfficial commented 2 years ago

@dt-zero Having some trouble using this on an Apollo Lake system (UP Squared), with the ME variant Intel seems to call CSTXE v3. This is an IFWI platform, which I guess explains "The ME region in this image has already been disabled" even though it's a stock image that the ME is clearly working when using. Using me_cleaner from your branch:

me_cleaner.py -s UPA1AM61_from_up_website.bin -O UPA1AM61_me_cleaned.bin
Full image detected
The ME region in this image has already been disabled
The HAP bit is NOT SET
Setting the AltMeDisable bit in PCHSTRP10 to disable Intel ME...
Traceback (most recent call last):
  File "/home/ethan/coreboot_for_up_squared/mecleaner_dtzero/me_cleaner/me_cleaner.py", line 915, in <module>
    pchstrp10 |= (1 << 7)
NameError: name 'pchstrp10' is not defined

For what it's worth, the other version you linked in this thread did not work either, though that's to be expected (it's not, as far as I can tell, designed for Apollo Lake, as that's not what the other user's problem was with).

I have tried setting the AltMeDisable bit in the flash descriptor with ifdtool, but this (rather expectedly) does not work - the ME is still alive and well, according to intelmetool (the version from this branch, Reddit post links to the GitHub but the GitHub has almost no info about it so linking to the Reddit post, needs to be used on this hardware, the one in coreboot master does not detect the newer ME) and Intel TXEInfo.

I hoped to try patching your branch for Apollo Lake myself, and, hypothetically speaking, were I to have extracted the contents of the Intel FIT binary for my TXE version (CSTXE System Tools v3 r12 Flash Image Tool, Linux version) using binwalk -e and then gone through all the XML-like files inside, I might have found that nothing resembling PCH_Strap_CSME_CSE_CSME_Reserved appears to be in there, and may then have uploaded an archive of the extracted files, which I may then have agreed to remove if you so desire.

Positive Technologies notes that Apollo Lake ignores the bup_not_temporary_disable flag, but they do not say that it ignores the HAP bit, which they specifically note the flag they call bup_not_temporary_disable is not the same thing as, so I strongly suspect there IS a way to set AltMeDisable/HAP on Apollo Lake - as you say, "I think it is unlikely that any ME versions lack the HAP bit behaviour".

I have attached the original BIOS dump (a raw, 16MiB whole-chip image), though it is also available from the manufacturer's website.

up_squared_original.rom.zip

UPDATE: I have noticed that ifdtool has a -p aplk "platform-specific quirks" mode for Apollo Lake. Running with this, it tells me that the image I previously disabled the ME on (seemingly ineffectively) has not had the ME disabled (when running with -p aplk, it calls it HAP bit not AltMeDisable), so I decided to try modifying the stock image again with -p aplk. I confirmed that there is only one bit difference between this second modified image and the original, so it seems like it's correctly set whatever it thinks is the HAP bit, however, when this is done to the stock BIOS, the machine refuses to boot. I'm going to consider this an improvement as opposed to doing nothing at all (as is what happens when you run without -p aplk, where the ME is obviously still alive), but obviously not usable. With Coreboot (I also have a Coreboot build for this machine) it works with or without the HAP bit, but the MEI PCI device does not appear with or without the HAP bit set, and setting it does not appear to affect anything in the Coreboot logs. As such, I still have no idea how to set the HAP bit/AltMeDisable on this machine, and the one way I think might set it properly (ifdtool -p aplk) breaks the only firmware I have for the machine that allows me to check if the ME is alive.

dt-zero commented 2 years ago

@DodoDude700 : You seem to have done your homework regarding what is out there, good job on that. Unfortunately, I have a pretty tough time with this one. In general, I don't see this PCH strap as part of the TXE platforms (Apollo Lake/Broxton/Gemini Lake) so I have no clue about whether or not it really exists for your device. I know this is not what you wanted to hear - sorry. My guess on ifdtool is that it probably has no clue about the actual bit layout either and it accidentally set something that causes failure during platform init. If that is the case, I'm not surprised, it's easy to misconfigure something in that region and have an unpleasant result.

DodoidOfficial commented 2 years ago

So, I spent some time talking to the Coreboot developers about this (in the IRC). As it turns out, the Apollo Lake TXE is quite markedly different from the Management Engine's other iterations. For example, to quote developer icon:

I can have a look at the docs, but have little hope. APL's CSE is not much like the usual ME. given how this CSE works, I wonder anyway what it is exactly that you want to disable

it has to load the x86 bootblock into SRAM for instance, and then take the x86 out of reset. that's unique to APL and its relatives. I don't know if there is a "do the x86 bring up but then shut down mode"

and in response to my question about ifdtool in APL quirks mode:

looks like strap[0] bit 16 is part of a field to configure something voltage id related (SVID_RAIL3_ID). you cannot trust ifdtool btw. changes there are rarely reviewed.

As such, it does not appear that Apollo Lake (and possibly "its relatives", maybe Gemini Lake too?) really have a HAP bit, at least not one icon was aware of or one I could find.

However, talking to Coreboot developers did solve one mystery - that of the Management Engine showing up under the stock firmware but not under Coreboot. In the coreboot sources, under src/soc/intel/apollolake/cse.c, the function disable_heci1 (where HECI is Host Embedded Controlled Interface, the interface to the ME) is used to disable communications with the ME after Coreboot is done dealing with it. Commenting out the contents of this function (leave the function itself, it still gets called, you just don't want it to do anything) and recompiling, the MEI shows up under Coreboot and we're able to look into its state a little bit more. Once again, just like stock firmware, at present, the exander77 branch needs to be used.

On the stock firmware, my intelmetool output (minus the PCI device "debug spam" the exander77 branch outputs):

MEI found: [8086:5a9a] Celeron N3350/Pentium N4200/Atom E3900 Series Host Bridge

ME Status   : 0x80000255
ME Status 2 : 0xf40400

ME: FW Partition Table      : OK
ME: Bringup Loader Failure  : NO
ME: Firmware Init Complete  : YES
ME: Manufacturing Mode      : YES
ME: Boot Options Present    : NO
ME: Update In Progress      : NO
ME: Current Working State   : Normal
ME: Current Operation State : M0 with UMA
ME: Current Operation Mode  : Normal
ME: Error Code              : No Error
ME: Progress Phase          : ROM Phase
ME: Power Management Event  : Clean Moff->Mx wake
ME: Progress Phase State    : (null)

ME: Extend Register not valid

ME: Firmware Version 3.1.2351.75 (code) 3.1.2351.75 (recovery) 3.1.2352.75 (fitc)

ME Capability: Full Network manageability                 : OFF
ME Capability: Regular Network manageability              : OFF
ME Capability: Manageability                              : OFF
ME Capability: Small business technology                  : OFF
ME Capability: Level III manageability                    : OFF
ME Capability: IntelR Anti-Theft (AT)                     : OFF
ME Capability: IntelR Capability Licensing Service (CLS)  : ON
ME Capability: IntelR Power Sharing Technology (MPC)      : ON
ME Capability: ICC Over Clocking                          : OFF
ME Capability: Protected Audio Video Path (PAVP)          : ON
ME Capability: IPV6                                       : OFF
ME Capability: KVM Remote Control (KVM)                   : OFF
ME Capability: Outbreak Containment Heuristic (OCH)       : OFF
ME Capability: Virtual LAN (VLAN)                         : ON
ME Capability: TLS                                        : OFF
ME Capability: Wireless LAN (WLAN)                        : OFF

Despite the concerns of some of the devs I spoke to that intelmetool may not handle Apollo Lake properly, on the stock firmware, this looks like a sensible output - we see that the TXE is alive and well in a normal state, that it's on firmware version 3.1.2351.75, and that its listed capabilities match what would be expected of the Apollo Lake TXE (while lacking the features of higher-end ME variants which Apollo Lake is not supposed to support. It should also be noted that these results agree with what Intel's own TXEinfo, which definitely DOES support Apollo Lake properly, says when run on the same machine. As such, I'm willing to consider intelmetool, at least this branch, a relatively decent way to determine TXE state on Apollo Lake.

Now here comes the weird part: under Coreboot, even without any kind of HAP/AltMeDisable bit set, and even with no modifcations to the ME firmware built into the Coreboot image's IFWI, intelmetool gives me this:

MEI found: [8086:5a9a] Celeron N3350/Pentium N4200/Atom E3900 Series Host Bridge

ME Status   : 0x80003052
ME Status 2 : 0x308a0040

ME: FW Partition Table      : OK
ME: Bringup Loader Failure  : NO
ME: Firmware Init Complete  : NO
ME: Manufacturing Mode      : YES
ME: Boot Options Present    : NO
ME: Update In Progress      : NO
ME: Current Working State   : Recovery
ME: Current Operation State : M0 with UMA
ME: Current Operation Mode  : Normal
ME: Error Code              : Image Failure
ME: Progress Phase          : Policy Module
ME: Power Management Event  : Clean Moff->Mx wake
ME: Progress Phase State    : 0x8a

ME: Extend Register not valid

ME: has a broken implementation on your board with this firmware
ME: timeout waiting for data: expected 8, available 0
ME: GET FW VERSION message failed

This is extremely interesting. The Management Engine appears to be in an intermediate, "half-disabled" state. As with me_cleaner'd firmware, we see we see Firmware Init Complete at NO. Like "traditional", non-AltMeDisable-based, no -S/-s me_cleaner, we see Error Code at Image Failure instead of No Error (as you would expect to see with AltMeDisable). However, what's going on does NOT appear to be akin to traditional me_cleaner - whereas you would expect intelmetool on me_cleaner'd firmware to give you Progress Phase: BUP Phase and Progress Phase State: M0 kernel load, here we have Progress Phase: Policy Module, which is a state I've never seen intelmetool give me, with any modified or unmodified ME. What's even weirder, Progress Phase State is 0x8a, which intelmetool does not appear to even have a name for, and Current Working State is Recovery.

We can also see that, whereas the ME is able to return its firmware version to intelmetool under the stock firmware, here that times out and causes intelmetool to exit. Patching out the firmware version check, we get a similar error in its place when it attempts to check capabilities:

ME: has a broken implementation on your board with this firmware
ME: timeout waiting for data: expected 5, available 0
ME: GET FWCAPS message failed

Seeing this, I very much expected the 30 minute watchdog to reset the machine, as the ME is clearly in a bad state, and this is not a state I have seen documented with any technique to disable the ME, so I did not expect it to work around the watchdog. However, I have left the machine running for hours in this state and it appears stable and reliable.

As such, it appears that Coreboot on this hardware has somehow broken the Management Engine/TXE firmware or forced it into recovery mode, preventing it from initializing properly but somehow either also breaking the watchdog timer or remaining alive enough to reset it.

I thought that Intel's own TXEinfo may be more enlightening about the TXE's state, but it fails with this:

Intel(R) TXEInfo Version: 3.1.70.2340
Copyright(C) 2005 - 2019, Intel Corporation. All rights reserved.

Error 83: Communication error between application and Intel(R) TXE module (HCI client)

Error 81: Internal error (Could not determine FW features information)

As such, I conclude that:

For my purposes, I think I'm now comfortable concluding that the Management Engine on this board is essentially disabled, albeit in a different state than other ME disable methods, which does not appear to have been documented before. Further research into this may be useful.

joex2021 commented 2 years ago

Hello,

I need a help regarding the following issue. I Have an Asus Z590-A motherboard with intel i7 11700k and intel Me 15.0.1466. I tried to disable ME with the me_cleaner made by @dt-zero and it didn't work, the same version worked on my old system that had ME12. Here is me_cleaner output i get when i try both -c and -s:

Full image detected
more than one $FPT found

here I attach the bios dump. z590_2.zip

Thanks in advance.

helios-chad commented 2 years ago

Hi, can you tell me if it is still possible to enable HAP on 11th gen? I want to buy new Hardware but I can't find any information about the intel ME stuff that is not horrendously outdated. Thx In advance.

joex2021 commented 2 years ago

I tired to enable it with -s command and it didn't work. You can download my BIOS dump and test it. I hope that @dt-zero can help us with this issue.

dt-zero commented 2 years ago

Okay, so since this pull request got a lot of traffic since I opened it, let me say something now. I love how many of you are enthusiastic about having a less proprietary and more open platform, I'm really glad to see that.

At the same time, my expectation when I opened this pull request was that it may or may not be merged but that it's going to be a reference point for @corna to help with the maintenance of the project. Since then, this grew to the size that it's taking up significant chunks of my time to keep supporting each and every one of you who reaches out on github or via email. I did this research on my own time but otherwise I'm full time working for a company (Zero IT Lab) that requires me to do a ton of other stuff during the daytime. It's becoming hard to balance that and it's unsustainable for me to dedicate significant free time to keep up with this.

I'll try to answer this set of questions that piled up until now (in my next comment), but please don't "expect" my support. Realistically speaking, since the project seems abandoned by corna, it's probably time for a fork/rewrite. It could be a company project for us, if there was financial backing to do it at reasonable market rates; but that kind of interest does not seem to be there. From my experience, most of the people who reach out to me are private individuals who do this for the enthusiastic/educational aspect. (and therefore don't want to/can't afford that kind of backing) If that ever changes, we'll probably pick the project up; otherwise unfortunately I cannot dedicate more of my time.

dt-zero commented 2 years ago

@joex2021 The HAP offset for your rom is 0x196. 0x0 = Hap disabled; 0x1 = Hap enabled. Make a copy of your bios rom and edit it like so

printf '\x01' | dd of=z590_copied.bin bs=1 seek=406 count=1 conv=notrunc

It will edit the file ("z590_copied.bin") in place and enable the HAP bit. Flash and you should be good. (No guarantees)

@helios-chad Yes, it is possible. As I showed above, we are still enabling the HAP bit on ME 15 which is more up to date. I haven't had any issues as long as you are on a consumer (CSME) platform and not TXE like @DodoDude700.

joex2021 commented 2 years ago

@dt-zero thank you for trying help us here, I understand all what you wrote and I totally agree with you. I have been following this project since I head about the ME dangers back in 2018 and I watched your video about intel ME, it is really sad that this great project is dying, I wish I had that advanced knowledge to help. Can you explain lite bit more how to edit the BIOS rom? Do I need a special tool to open and edit the BIOS rom?

Thank you for your help and patience.