corna / me_cleaner

Tool for partial deblobbing of Intel ME/TXE firmware images
GNU General Public License v3.0
4.46k stars 277 forks source link

Complete ME disablement on X79 systems? #278

Open ghost opened 5 years ago

ghost commented 5 years ago

Hi all, It seems that some chipsets (or mainboards?) do not have an active ME watchdog timer. I have an ASUS Rampage IV Extreme and was able to wipe the ME region (all 0xFF) with almost no ill effects:

# ./intelmetool -s
MEI not hidden on PCI, checking if visible
MEI device not found

# lspci | grep -c Comm
0

# uptime -p
up 19 hours, 14 minutes

# flashrom -p internal -r /tmp/no_me.rom &>/dev/null
# ./ifdtool -x /tmp/no_me.rom >/dev/null && hexdump -C flashregion_2_intel_me.bin
00000000  ff ff ff ff ff ff ff ff  ff ff ff ff ff ff ff ff  |................|
*
00001000

AFAICT there are only two side-effects of removing the ME firmware:

If anyone else has this board, other X79 mainboards, or even more recent intel HEDT systems (X99 or X299), I think everyone would be interested in finding out whether they have active watchdog timers.

weareanomalouswearearegion commented 5 years ago

Do you mean WDT? My Rampage IV Gene has it(refer to picture).

I successfully ran ME_Cleaner with the /S flag on it, and no 30 minute reboots. Everything works fine AFAIK.

Did you wipe the ME region more completely than the ME_Cleaner script or something? I thought it is necessary for ME to initialize the CPU in the first place? Intel WDT

ghost commented 5 years ago

Do you mean WDT?

I don't think so. I couldn't find those IO ports in the X79 datasheet, but I think they are part of the southbridge watchdog. The ME watchdog isn't exposed to the OS AFAIK. The only way to check whether it exists is to erase the ME region, boot the board, and wait 30 minutes.

Did you wipe the ME region more completely than the ME_Cleaner script or something?

Yes, I erased the entire ME region. Normally when you do this the ME will shut off the system after 30 minutes, but apparently that does not happen on some machines.

I thought it is necessary for ME to initialize the CPU in the first place?

I heard this as well but it doesn't seem to be the case for X79/SandyBridge-E. Maybe it is true for more recent Intel systems or non-HEDT platforms?

If you want to test this I can prepare a ROM for Gene or give instructions for creating one. You will need to be able to recover from a bad flash if it doesn't work. Gene seems to have USBFlashback so that shouldn't be a problem.

weareanomalouswearearegion commented 5 years ago

I have a CH341a flasher, so I'm not worried about a bad flash. I don't mind testing it, but I can't deal with a broken LAN port, so I might need to switch back to my current image after testing.

As for modding the BIOS, I already have a modified image(Microcode update(42E), NVMe mod, ME_Cleaner), so if you like me to test removing the Intel ME region entirely, I would prefer instructions for further modifications instead(I would still like to keep my current modifications in addition to any new modifications required).

As for your first side-effect of rebooting after resetting BIOS, I can confirm I encountered the same after using ME_Cleaner with the /S flag.

The onboard NIC doesn't work on a cold boot. This can be fixed by resetting the NIC: (volatile uint32_t)GBE_BAR |= 0x84000000;

Do we put this line somewhere in the UEFI or something? Or is this a Linux command? I'm currently using Windows, at least till my NVMe adapter comes (Well, not like I'm proficient with Linux in the first place). If there is an automatic way of resetting the LAN after a cold boot I don't mind using the modified BIOS on a long-term basis.

PS Did you actually manage to flash the image with ME removed using BIOS flashback? I tried modifying the ME region and the flash seemed to pass (the button flashing and all), but the modifications were not actually reflected in the system (I did the NVMe mod+Microcode update(42C)+ME_Cleaner all at one go, and none were reflected). I know a simple Microcode update works however because I updated the microcode (to older 42A) using USB flashback. In the end I tried using FTK8 to flash but it only wiped the SPI chip and bricked my board. I only managed to get it back up and running after getting an SPI flasher, but lost the UUID permanently in the process(not like it matters however).

ghost commented 5 years ago

I already have a modified image(Microcode update(42E), NVMe mod, ME_Cleaner)

In that case I assume you are already familiar with UEFITool? Here's what you need to do:

All of this assumes you have not used the -t flag of me_cleaner to truncate the ME region. In that case you will need a differently-sized file. You can create the file by extracting the ME region with UEFITool and overwriting the entire contents with 0xFF bytes in a hex editor.

After this, if you boot up your board and it stays on for more than 30 minutes, Gene doesn't have an active ME watchdog timer either.

Do we put this line somewhere in the UEFI or something?

That's C pseudocode. I wrote a UEFI module that does this but it also takes over the BDS process and boots directly to GRUB (in firmware). I can see about isolating the GbE fix into a separate module to be inserted with UEFITool. The fix can also be done in GRUB or in Linux, but in the latter case you have to reload the e1000e module. I'm not sure about Windows.

PS Did you actually manage to flash the image with ME removed using BIOS flashback?

Yes, USBFlashback behaves differently depending on the file name you use. The normal file names like "R4E.CAP" and "R4G.CAP" only flash the BIOS region. "ERALL.CAP" seems to flash the BIOS and ME regions at least. You can also use the ROM variants of all of these names if you don't have Capsule files.

Thanks for testing this out :+1:

weareanomalouswearearegion commented 5 years ago

Managed to remove the ME region entirely. Fingers crossed for the 1/2 hour mark! Countdown

weareanomalouswearearegion commented 5 years ago

...and, it works!

Interestingly, Intel ME completely disappears from device manager, instead of showing a power failure status.

Does this, by any chance, hint at Libreboot potential?

Success ME_Missing

weareanomalouswearearegion commented 5 years ago

Interesting findings from my side:

1: Not sure if it's fast boot or not using Windows Boot Manager(boots directly off SSD), but I don't experience the LAN port disablement upon cold boot.

2: Now, this shows up during boot: 20190529_181919

3: Also, there is now an additional entry in the boot menu: 20190529_181358

Now, in contrast of using ME_Cleaner, this actually resulted in the motherboard to post very slowly each time it starts, instead of just the first one.

Will update if and when new findings are spotted.

Edit #1: Intel WDT is now gone and changed to "Motherboard Resources": WDT_missing

ghost commented 5 years ago

Does this, by any chance, hint at Libreboot potential?

I think so! If the hardware initialization bits can be reverse-engineered the only proprietary parts would be intel microcode & iROG (EC) firmware, both of which are already issues for libreboot-supported systems like the X200.

I don't experience the LAN port disablement upon cold boot.

Interesting. That could be a side-effect of the PXE option rom, which seems to have been enabled on your system for some reason. It could also be that the Windows drivers are able to recover from the failure state while the Linux drivers can't.

this actually resulted in the motherboard to post very slowly each time it starts, instead of just the first one.

That's unfortunate, but something I also noticed when testing this. I think it may be caused by the ME drivers needing to time out before the post continues. I'll need to look into this more.

Intel WDT is now gone and changed to "Motherboard Resources"

I didn't expect this. Maybe those ports do have something to do with ME.

weareanomalouswearearegion commented 5 years ago

Well, after finally shoving the motherboard into the PC case for general use, I have an update here. Intel ME intermittently(persistent until reboot) appears and disappears in Device Manager: Update1

I have confirmed that nothing has modified the BIOS to add the ME modules. I used FTK to dump the BIOS and used UEFITool to verify, and the ME Region is still empty: Update2

I noticed that sometimes, Windows load time is significantly faster(insanely fast (<5 seconds) probably due to Fast Startup, but is somewhat sluggish from Ctrl-Alt-Del prompt to password prompt). In this case, I noticed the ME appears but "cannot start". However, if Windows loads slowly(typically from restart/power loss)(the loading speed does not necessarily reflect the capabilities of a SATA SSD), ME disappears completely from the system as per normal. The long POST time is consistent.

I suspect this is due to Fast Startup, as I just attempted to shutdown+power on and Windows load up fast due to a quasi-hibernation technique, and ME appears albeit broken. A restart from Windows got everything back to normal(ME disappears).

In HWiNFO64, the Intel Watchdog still shows as "Motherboard Resources" consistently. I also have no problems with LAN at all as well.

Not sure if this is normal/expected. Could there be hidden ME modules in other regions of the BIOS which caused this? Can ME even "load partially" if the ME region in the BIOS is empty?

(Edited to remove/rectify inaccurate information)

weareanomalouswearearegion commented 5 years ago

Another thing: Not sure if this is the watchdog shown in HWiNFO64, but this device is disconnected from the system after removing ME (Both completely and after ME_Cleaner). It never came back.

Mobo_rsrc

Interesting find: The LAN chip is now detected as the "enterprise" 82579LM instead of "Consumer" 82579V after removing ME completely(but not after ME_Cleaner).

LAN Model Changed

Edit: Did ME_Cleaner and ME Removal on same day. Managed to trace back the time I did them by comparing time commented on GitHub and the Last Arrival Date in Device Manager.

ghost commented 5 years ago

Could there be hidden ME modules in other regions of the BIOS which caused this? Can ME even "load partially" if the ME region in the BIOS is empty?

I don't think the ME is working at all. I was able to get the MEI device to come back on warm boots but it isn't functional - it returns garbage in lspci:

00:16.0 Communication controller: Intel Corporation C600/X79 series chipset MEI Controller #1 (rev ff) (prog-if ff)
    !!! Unknown header type 7f
    Kernel modules: mei_me

I think this is just a sign of the BIOS taking different paths for cold vs. warm boots. I have an idea about preventing this but I will need to test it.

RE: sluggishness, I don't experience it on Linux, and I don't know why it would happen on Windows. I would try uninstalling the Intel Watchdog Timer and Intel MEI drivers, if you have them installed.

The long POST times are also not something I've been able to reproduce easily. They have happened to me but are rare and it's difficult to test fixes as a result. Can you give me an idea of how much longer your POSTs are without ME? Are there any POST codes that stand out?

BTW, I found that I could get the "Intel Boot Agent" screen to appear by enabling "Intel LAN PXE OPROM" in the "Advanced->Onboard Devices Configuration" menu of the BIOS setup. Maybe check that?

weareanomalouswearearegion commented 5 years ago

Here's the post code video(downscaled to 480p) of the mobo under "warm boot"--ME appears broken in Device Manager: https://gofile.io/?c=79wNkf (Not sure if there is a better place to upload this. Also, might need to download the file to watch; it does not stream in chrome properly on my laptop for whatever reason)

During a warm boot, the POST code ends with "40". The uppermost LED is permanently lit after POST. Time taken from power on to successful POST is ~40 seconds.

Here's the post code video under cold boot--ME is not present: https://gofile.io/?c=ETqIHT

During a cold boot, the POST code ends with "AA". The uppermost LED is also permanently lit after POST. Time taken from power on to successful POST is ~40 seconds.

Approximate time (1 trial each, not average of n trials) taken from POST success to Windows Logon (Ctrl-Alt-Del prompt):

Cold--2 minutes (~5 seconds ROG logo+IBA, 1 minute 53 seconds on Windows startup scroll, 2 seconds to lock screen)

Warm--14 seconds (~5 seconds ROG logo+IBA, <1 second on startup scroll, 8-9 seconds to lock screen)

Approximate total time taken from power on to Windows Logon (taken from separate singular trials):

Cold-- 2 minutes 40 seconds Warm-- 54 seconds

Will try cold boot with ME/Motherboard Resources disabled in Task Manager, and investigate on the LAN PXE OPROM on next post.

BTW, did enabling the OPROM fix your LAN issue during cold boot? Also, are you running on UEFI mode or Legacy? I'm currently running Legacy boot+MBR, but will need to switch to UEFI mode+GPT after the PCIe NVMe adapter comes in the mail.

weareanomalouswearearegion commented 5 years ago

Uninstalled the ME drivers and rebooted. POST success to Logon Screen took merely 33 seconds, which is much faster for a cold boot. However, Intel ME is back in device manager!...for a short while. It disappears shortly after in device manager.

Meanwhile when ME appeared briefly, the following still holds true: LAN recognised as 82579LM Watchdog still recognized as "Motherboard Resources" in HWiNFO64 ME is still absent altogether in HWiNFO64

The briefly-shown ME in Device Management:

ME_Appearance

Another relevant screenshot, after it disappears: timestamp

When I reboot again without uninstalling the drivers, the system took 2 minutes to boot again(sans POST). ME never appears in Device Manager at all, just like before.

Therefore, I can conclude that uninstalling the driver does help(loading Windows, not solving the POST duration), although the drivers needs to be uninstalled before each cold boot. This could mean Windows is still detecting the Management Engine/whatever remains of it during boot even without the drivers.

ghost commented 5 years ago

My times for comparison:

Original firmware, including ME: Cold boot: 32.4 seconds Warm boot: 25.6 seconds

Firmware with ME removed: Cold boot: 32.6 seconds Warm boot: 37.6 seconds

So it seems removing the ME increases warm boot times by ~12 seconds on my board. My guess is that this is caused by the broken MEI device reappearing and firmware MEI drivers failing to interact with it.

are you running on UEFI mode or Legacy?

I boot in UEFI mode with the CSM disabled, though these tests were done with the CSM enabled. If you don't need the CSM after you get your NVMe adapter I recommend disabling it, as it drastically increases POST times.

did enabling the OPROM fix your LAN issue during cold boot?

No, so I'm thinking either the Windows drivers are able to work around the issue or it doesn't affect Gene. The former could explain your 82579LM "upgrade".

ghost commented 5 years ago

As suspected, my fix for the MEI device reappearing also fixes the POST delays, with warm boots returning to their normal 25 seconds with CSM on. (9 seconds with it off) Hopefully it will prevent Windows from reinstalling the MEI drivers as well.

I've patched the modules for Gene if you want to test the fix. You need: Gene_Recovery_MEPlatformPEI.bin and Gene_MEPlatformPEI.bin

Open your ROM in UEFITool and replace the modules as shown:

gene_mei_patch

weareanomalouswearearegion commented 5 years ago

More updates on my side:

"Intel LAN PXE OPROM" is shown as disabled in BIOS. Yet, I see that IBA prompt during startup.

I managed to take a retired SSD with Windows 10 1809(meaning released ~September 2018)(previously it was 1607, meaning released ~July 2016) and boot it up (UEFI mode). Initially it was reasonably fast to start up(sans POST), but after installing/re-configuring the drivers and a cold boot it detected the ME and again took ~2 minutes to load to the desktop(No password set on this install). I did not install the ME driver and all the drivers were installed from device manager itself. In this newer build of windows, the LAN is also detected as the Enterprise version.

Gonna mod my BIOS right now, so fingers crossed.

weareanomalouswearearegion commented 5 years ago

Changed the modules, flashed the new BIOS...and the motherboard stopped POSTing. It's the same symptom as the time I bricked the board with FTK, whereby it just cycles on and off repeatedly.

After I replaced the first module with the recovery PEI file, I noticed the bottom 2 volumes(8C8CE...) were also labelled "Rebuild". After changing both all the bottom 4 volumes have "Rebuild" status on them.

I have verified the flashing went correctly and also re-modded my BIOS to the same result.

Could there be an issue with the 2 modules?

ghost commented 5 years ago

Ouch. I'm sure you know how to recover.

I noticed the bottom 2 volumes(8C8CE...) were also labelled "Rebuild".

This shouldn't be happening. Here's what it looks like for me: gene_mod

Your ROM is based on 4901 right? Can you post screenshots of the Information panel in UEFITool with the PE32 section selected? Modded and unmodded preferably.

EDIT: I was using an older version of UEFITool. I see the volume rebases now. Still investigating.

weareanomalouswearearegion commented 5 years ago

Screenshots below:

Before_replacement Replacement_Backup Replacement_Both

5 6

While my motherboard underwent cycle-on-cycle-off process, there is still a flash of POST code on the top right instead of being blank (When I accidentally erased the boot block with FTK), which probably implies the BIOS image is partially working.

ghost commented 5 years ago

Your MEPlatformPEI modules are at a different address than I expected. I like to avoid having UEFITool relocate PEIMs, as it always seems to cause problems. If you extract the original PE32 sections (Extract body) and upload them I can apply the patch.

ghost commented 5 years ago

Extract body, just edited

weareanomalouswearearegion commented 5 years ago

folder.zip ...and uploaded.

ghost commented 5 years ago

...why am I using dropbox when I can just attach zips.

Here you go: modules.zip

weareanomalouswearearegion commented 5 years ago

And...

It works like a charm! ME does not show up 'hidden' in cold boots anymore, and the system stopped taking so long to POST! ~48 seconds from power on to Windows on cold boot. As for POST, it takes approximately 10 seconds for a cold boot, which is a big improvement.

LAN chip still detected as 82579LM, and I still don't experience LAN issues on cold boot. The IBA screen still appears with the PXE OPROM disabled.

As for soft boot, it is also faster (~20 seconds from power on to lock screen) thanks to the reduced POST times. The ME is gone for good this time round.

If you don't mind me asking, what did you modify on those modules? Not like I would understand much about how this works but...

ghost commented 5 years ago

It works like a charm!

Great!

what did you modify on those modules?

I changed them to OR the Function Disable 2 (FD2) register with 0x1E instead of 0x1C. Normally the MEPlatformPEI module only disables the second MEI device. I add bit 1, which disables the first MEI device as well. It's a simple one byte change:

Screenshot from 2019-06-07 14-27-58

And the relevant southbridge documentation: Screenshot from 2019-06-07 14-21-09

Without the MEI enabled the other modules (and Windows) don't slow things down by trying to talk to a broken ME.

weareanomalouswearearegion commented 5 years ago

Mind blown

It's a simple one byte change

I don't even know where to begin lol. How do you find the correct module and the correct byte to replace? UEFITool/HxD surely doesn't show anything useful does it?

ghost commented 5 years ago

Well, I knew I was looking for FD2, and the southbridge documentation tells me it's at RCBA + 0x3428. I know that RCBA on these boards is 0xfed1c000, which means FD2 is at 0xfed1f428. Searching for that address in UEFITool gives a bunch of results, but I wanted to put the fix in a PEI module so it would run as early as possible. That leaves SBPEI, HECIPEI and MEPlatformPEI, and opening the latter in radare2 revealed a great spot to make the change:

r2

weareanomalouswearearegion commented 5 years ago

My NVMe adapter just came in the mail today, and I managed to install a fresh copy of Windows 10 (1809) onto the attached NVMe SSD. Even though it is a UEFI-based install, there is no apparent issues(except the SM2263XT-based "AliExpress" KingSpec SSD is having atrocious speeds all round, even with HMB enabled) and everything seems very stable. ME never appeared again since the latest modification of course. POST timings are still ~10 seconds but I'm not bothered by it.

The boot times are not improved at all...but I blame it on the SSD for this one:

SSD

Honestly speaking, I'm not even sure whether this excuse of an NVMe drive is any faster as compared to my Samsung 840 SSD I used prior. The Random Reads/Writes are simply horrid. Fortunately, a 256GB PM981 is on the way to replace this thing, and I might be able to report some improved numbers lol.

At this point, do you think it's a good idea to recruit more testers from other enthusiast/modding forums? This place is relatively quiet in comparison. Of course, there can be barriers by doing so, such as modding the PEI modules in DIY situations...

While removing the ME region is simple, the modding of the PEI modules could be challenging though--I still have almost no idea of what's going on, or how to find the correct part to replace.

Warning--Gore ahead

What I kinda figured out, or what I think I figured out but in reality I didn't:

...and pretty much nothing else. Wow.

What I still don't know:

Search

magic numbers

radare2

Trying to use this to open and examine the PEI modules reminds me of the pain of trying to install an IME on an old Solus VM 3 years back--a mixture of despair, confusion and running all sorts random commands blindly while following a guide for an seemingly unrelated issue. I somehow managed to succeed on that one, but not this time round. What am I even looking at here(I think I know it's Assembly Language of some sorts)? Unknown gibberish

...I'm not a Linux guy. Sigh.

ghost commented 5 years ago

At this point, do you think it's a good idea to recruit more testers from other enthusiast/modding forums?

Yes, I think so. I'm interested to know whether this is exclusive to Asus, exclusive to X79, or in the best case, affects other HEDT platforms as well. I'm not active on forums so I don't know where the best place to post it would be. I'll create a repository to organize this (one github issue per board) so as to not clutter me_cleaner's issues.

Of course, there can be barriers by doing so, such as modding the PEI modules in DIY situations...

Agreed. I think UEFIPatch can help in automating this kind of thing. I'm not sure that all X79 boards use the same module though, so multiple patches may be needed.

The rationale for changing from 0x1c to 0x1e...

Exactly. It's interesting that bit 3 is set here, as it's marked reserved in the datasheet. The usual practice for reserved bits is to not change them.

How to find the RCBA of a board if the board is not part of "these boards"

RCBA (Root Complex Base Address) is configured in a southbridge register. For X79, this is register 0xF0 of the LPC Bridge (Bus 0, Device 31, Function 0, or 00:1f.0). So we can find out the current RCBA by reading that register from the OS. In linux, this can be done with the setpci command:

# setpci -s 00:1f.0 0xf0.l
fed1c001

Bit 0 is not part of the address, it just enables decoding of the memory range, so RCBA is 0xfed1c000. If there is a Windows tool that lets you inspect PCI configuration space you could also use that.

How to search address 0xfed1f428 in UEFITool. I don't get any results doing this:

x86 is little endian, so when you search for a memory address you need to reverse the order of the bytes. You need to search for 28f4d1fe.

I managed to make the magic number show up somehow, but have no idea how to edit it, nor where to edit it.

r2 definitely has a steep learning curve. I would still consider myself a beginner. To edit in r2 you have to open the file in read/write mode, which you can do by starting radare like "r2 -w path_to_file". After that, in visual mode, you have to position the cursor on the byte you want to edit and use the 'wx' command to overwrite it. In this case you would use "wx 1e". That said, for a one byte change it may be easier to just use a hex editor.

I think I know it's Assembly Language of some sorts

Yes, it's x86 assembly language. More specifically IA32 assembly, as all of the PEIMs are 32-bit.

ghost commented 5 years ago

The testing repo is https://github.com/nkht/me_removal

If you know some good places to post it let me know, or feel free to post it yourself if you are already active in a forum.

Nowaker commented 5 years ago

BTW guys, regarding the e1000e having troubles, it might not be Intel ME related at all. I've recently encountered an issue with my Intel GbE being totally unusable on Linux while it worked fine on Windows. https://bugs.archlinux.org/task/62699

weareanomalouswearearegion commented 5 years ago

@nkht Just an update, I have posted this in multiple subreddits. Here's the one that made the most traction: https://www.reddit.com/r/hardware/comments/bywtib/complete_removal_of_intel_me_firmware_on_certain/

Considering suitability posting on modding forums such as win-raid and bios-mods forums right now

ghost commented 5 years ago

@Nowaker I think this is a separate issue. In my case when the NIC is broken e1000e fails during initialization and the device doesn't even appear in ip link

@weareanomalouswearearegion Thanks!

Nowaker commented 5 years ago

@nkht Alright! Mine showed up but couldn't send any packets (e.g. to obtain an IP from DHCP).

weareanomalouswearearegion commented 5 years ago

Posted on TPU & ycombinator forums as well.

Unfortunately bios-mods and win-raid have relatively few people who wants to remove the Intel ME, and instead it's often the other way round (fixing a broken ME). Probably won't have much uptake there, but I might still post there anyway though.

@nkht

Also, I came across this discussion: https://forums.anandtech.com/threads/what-controls-turbo-core-in-xeons.2496647/page-86#post-39245371

Apparently on some X99 platforms, the Intel Microcode can also be removed from the BIOS after ME_Cleaner, with the consequence that S6 will be broken and require disablement. Does this imply that systems with removed Intel ME have the same capability? This could mean further compatibility with Libreboot, which requires/prefers the microcode be removed from the BIOS. I'm not sure about the actual benefits of removing the microcode however, since it is required for mitigations against speculative execution.

ghost commented 5 years ago

I don't think the ME state is related to microcode updates on X79. I had actually been running my system without microcode updates for a while, but decided to install them to fix the speculative execution vulnerabilities. I didn't notice any issues when running without them, but that will depend heavily on your CPU model, stepping, and what features you need. My CPU for reference:

model name      : Intel(R) Core(TM) i7-3960X CPU @ 3.30GHz
stepping        : 7
microcode       : 0x710

Ultimately you will be running microcode either way. Removing the updates just downgrades the version.

Unrelated - I wrote that UEFIPatch file to automate the MEPlatformPEI change. It's in my me_removal repo if you're interested.

weareanomalouswearearegion commented 5 years ago

Well, I'm just curious of the forum post regarding the use of ME_Cleaner followed by removing the Microcode completely from the BIOS to obtain the “IntelRCSetup" tab in the BIOS of certain X99 boards. The complete removal of the Microcode in addition to the ME firmware in the SPI chip might interest Libreboot developers more, since they prefer the Microcode to be run from the CPU itself(as read-only). I would rather use the latest microcodes however, for protection against the various speculative execution attacks.

Will fiddle around with the patch file on some BIOSes later.

ghost commented 5 years ago

“IntelRCSetup" tab

Sorry, the images don't work for me so I missed that. It's a menu to configure the Intel reference code blob. From searching, it seems it's visible by default on some boards, so my guess is the me_cleaner/microcode situation is just a quirk of whatever board that user has. There are some interesting ME settings in there but I don't know if changing them is any better than running me_cleaner with the -s flag. It's definitely something to look at if any X99 owners show up.

Libreboot's microcode requirement should be fine - at least for some CPUs.

weareanomalouswearearegion commented 5 years ago

I'm not exactly sure of Libreboot's interest in this at this point to be honest; there isn't much activity in that subreddit (There are 2 mods who are also devs there so I doubt they didn't see this) and naturally there isn't much replies there. One comment also mentioned that the team is tight in resources and manpower as well.

I have tried your UEFIPatch file and it seems to work for the X79 motherboard firmwares I downloaded, including new Aptio 5 ones such as X79 Deluxe (Of course, I am not able to test whether the modded firmware are functional or not). However, trying to run it on an Asus C602 (Also patsburg PCH) (version 5802 for Asus Z9PE-D8 WS) motherboard firmware did not work (UEFIPatch gave the error along the lines of 'nothing to patch').

persmule commented 5 years ago

I believe that coreboot should be asked first for x79, even if you want Libreboot to support it.

Since Libreboot is a downstream project of coreboot, I do not believe that Libreboot might want to add new board(s), or even accept new featured patch on their own.

(I used to submit one for T400s, but it is ignored by Libreboot, so I eventually submitted it to coreboot and get it partially accepted. )

ghost commented 5 years ago

@weareanomalouswearearegion I couldn't find the MEPlatformPEI module in the Z9PE-D8 WS BIOS. That board will probably need a separate patch - if ME removal works.

@persmule Yes, coreboot would probably happen before libreboot, but I don't think either are within reach at this point. Coreboot doesn't support Sandy/IvyBridge-E, and adding that support means reverse engineering the hardware initialization modules from the original BIOS. I'm unfortunately not skilled enough to do that, and it seems too big a favor to ask of the {core,libre}boot devs.

weareanomalouswearearegion commented 5 years ago

@nkht My bad, I think I opened a wrong BIOS file. Searched it again and found out there indeed isn't a MEPlatformPEI module.

@persmule May I ask what exactly is a 'submission'? Is it just a request to the devs themselves or a 'you build partially/fully, they approve/enhance" situation?

persmule commented 5 years ago

@weareanomalouswearearegion Mainly "you build partially/fully, they approve/enhance". I used to send patches containing enhancement ("featured patch" mentioned in the context of my last comment) to devs, asking them to review and merge.

I hardly found any portal to contribute to Libreboot, so I has chosen emailing my patches to their mailing list, but Libreboot team ignored them. On the other hand, coreboot has their gerrit open for everyone to submit their patches to get reviewed and accepted.

I mean, if even "you build partially/fully, they approve/enhance" get ignored by Libreboot, how can they accept mere requests for new boards not supported by coreboot at the time?

I believe that the behavior above implies that Libreboot team have set their role as "deblobbing coreboot" only, so featured patches (including those adding new boards) as well as requests with similar goals should go to coreboot instead of Libreboot.

You may try to mail your request for X79 to both mailing list of coreboot and Libreboot, seeing which team will reply.

weareanomalouswearearegion commented 4 years ago

Well, giving up on an open source firmware for now, I have discovered another motherboard model that apparently does not have proper Intel ME modules/region in it. It is a thin Mini-ITX(not really, as it does not have the same power jack as Intel's specifications) motherboard sold at various sites such as eBay, Taobao and AliExpress under the name "HM65-988" without a manufacturer brand. Being 'new' boards, they are most likely made from salvaged 6 series mobile chipsets (and not necessarily HM65, contrary to what its name suggests). After obtaining one of these motherboards, which came with a QM67 chipset, I popped a dual core sandy bridge i7 in it, and discovered the following:

Intel Management Engine is not present in Device Manager: DeviceManager1

HWiNFO64 does not show Intel ME component under the motherboard section: HWiNFO64

Intel MEInfo does not detect ME as well, throwing an error similar to X79: MEInfo7

Using FTK8 to dump the BIOS, the structure of the firmware is unconventional. The motherboard is using an AMI UEFI implementation. Note the lack of ME region (and other regions): MePolicyInitPEI UEFITool1 UEFITool2 UEFITool3 UEFITool4

A random picture of the motherboard: Motherboard

There is only one module that has 'ME' in it, which is the MeInitPolicyPEI module. There is also a CpuInitPEI module present (maybe BUP is in here?). Is this conclusive evidence that ME is not actually present at all? If this is the case, could it be the case that all 6 series chipsets, and X79 (+all C200, C600 equivalent chipsets) have inactive Intel ME Watchdog timers?

If anyone wants to investigate further, I can dump the BIOS files here.

Stitch626 commented 4 years ago

https://github.com/corna/me_cleaner/issues/326

Stitch626 commented 4 years ago

I've tested me_cleaner on a MSI X79A-GD45 (8D) somewhere in 2019 and it was kind of working; me was dead, boot time was ok, no lan issues, but the ime device was shown as defunc instead of completely disappeared as on the h61, z77, etc boards I own. I thought the device may be hardcoded in the pci routing table, but this thread gave me some hope. Maybe I'll mess with the board again as I own two of them and also an external flasher :)

mrhappyhof commented 3 years ago

I have tested the procedure on a Asus P9X97E-WS. Everything is working fine

seniorm0ment commented 3 years ago

Has there been successful attempts at Corebooting either of those two ME removable X79 motherboards? (ASUS Rampage IV Extreme, ASUS Rampage IV Gene).

if not, could someone please attempt porting it please? It'd be beneficial to the community as these are the only boards we have to my knowledge with this support. I hope to do a ME Removed, Corebooted firewall build using a Xeon 2670 and one of these motherboards. The issue is lack of Coreboot support currently from what I can tell.

also what are the proper instructions to follow for ME removal on these? thanks.

Stitch626 commented 3 years ago

@weareanomalouswearearegion Even if there are no watchdog timers on the 6th gen boards, some of them can't be me-cleaned. Since 2019 I came across several 6th, 7th and 8th gen boards as they got cheaper. However, one of them had a BIOS and no UEFI at all. Only a little 3TB expansion OpRom. Once I disabled ME there, the board totally messed up. It stopped to properly cold boot, restarted randomly and even if it booted, it's been slow as hell. Had to re-flash the original bios. Even disabling the me instead of trying to remove modules (software disable) the same thing happened. Just sad. I thought it would be easy to remove the ME, given it's been a BIOS. How naive of me...

Another thing I noticed is: ME behavior changed a bit with the 8th gen boards. I have a asus one with h81(?) and after disabling the ME, CPU-Z failed to report any CPU-Clockspeeds. Same for other software. But it seems to still perform well, though. May test the same thing on a msi one.

Stitch626 commented 3 years ago

I can report success!

I have FF'd the ME region and applied the patch provided by @nkht

Flashed my MSI X79A-GD45 (8D variant, bios rev 12.7) with it, and it just works. The boot time even improved. I have tested:

Everything seems to work as expected and the MEI device disappeared without reappearance on cold/warm boots or after s3 resume. And no, I don't have the network boot agent issue. And the network card is still recognized as the correct model.

Anything else to test?

I may test the procedure on a ASUS P67 board, if anyone is interested :)