SuperThunder / HP_Z420_Z620_Z820_BootBlock_Upgrade

A guide and collection of resources on how to make 'version 1' HP Z420, Z620, and Z820 workstations compatible with Ivy Bridge processors.
MIT License
43 stars 7 forks source link

BIOS modding - NVME boot, etc: #13

Open bibikalka1 opened 3 weeks ago

bibikalka1 commented 3 weeks ago

@BillDH2k

OK, 3 BIOSes are attached for Z620. All are based on J96. All modding is done with "MMTool Aptio V4". The microcode downgrade is inserted manually, by replacing the microcode module padded to the same size (did not like how UBU tool shifted things around).

Most ambitious (try first) - NVME/ReBar/MC [NVME boot, ReBar PCIE, microcode downgrade for speed to the version prior to Meltdown and Spectre fixes] Next (benign) - NVME/ReBar Mild (known to work elsewhere although different versions) - NVME

Only this region (-A 0x580000 -L 0xA70000) needs to be flashed (out of the whole file), to the same region on the flash chip: fpt.exe -f bios_file_extract.bin -A 0x580000 -L 0xA70000

Don't flash the full file as-is, only flash the region above.

J61_0396_nvme.zip J61_0396_nvme_rebar.zip J61_0396_nvme_rebar_mc.zip

BillDH2k commented 3 weeks ago

@bibikalka1

1) Which local BIOS version should I start with? I have ME7+B2011, ME8+B2013 ("Manufacturing mode"), and ME8+B2013 ("normal").

2) Could I try these on a Z420 (which I has an easier access to)?

BillDH2k commented 3 weeks ago

@bibikalka1

Tested "nvme+ReBar+mc" on Z420, V1 1620 CPU (Note: I thought I was testing "nvme" previously, due to mixed up the extracted BIN files) . BIOS mod: V396/ME8/B2013 + 0x580000 region replacement. Performed "MEBLAST 8+V207" sequences to enter "MF" mode, flashed the whole bios mod via "fpt.exe -f".

Reboot results: Initially in power ON/OFF cycles by itself. Then eventually powered off. Did a CMOS clear. Came back with "Incorrect microcode detected, please contact HP" "1801-Microcode Update Error. Missing or invalid Processor Microcode."

Recovered with a flasher (not sure which CPUs it expect). I did test Crisis BIOS Recovery via jumper.

Update: I redid the test. This time, flashed the saved "working BIOS dump" first, then followed by flashing 0x580000 region. Here are the results. Previously, I combined the change first, and flashed as a single file.

Test 1 ("nvme" mod only): Rebooted successfully! Did not test the 'nvme' feature, but I saved the BIOS dump, for you to examine.

Test 2 ("nvme + ReBar"): Rebooted successfully. Saved the BIOS dump.

Test 3 ("nvme + ReBar + mc"): Just like earlier test. Ended up in power ON/OFF cycles, by itslef. Eventually power off (after 5-6x). Manually power on again, after several ON/Off cycles by itself, post to screen "Incorrect microcode detected ...".

Note: I was able to recover via Crisis BIOS Recovery jumper!

bibikalka1 commented 3 weeks ago

@BillDH2k

Awesome testing, many many thanks!!!

Z420 is a perfect substitute for Z620, the BIOS is same. And the correct plan too - reload full good BIOS, then the patched J396 code on top. Whichever machine is logistically easier for you to try - that's the one, and v1 cpu helps until we have the other hack working for v2 with MEBLAST!

Good to hear crisis recovery jumper working, I could not get it to go while still in manufacturing mode. But you took care of that by flashing a good BIOS copy first. But I hear crisis recovery is a tricky procedure taking as much time as a programmer if you are skilled with the latter?

Microcode patching is a mess, this is my 2nd try, I will work more on that. The other modules are great, need to test now if ReBar actually works as intended :)

P.S. I skimmed through your Hackintosh guide. Do you think some of the challenges can be addressed by BIOS patching, like USB3? The BIOS patching is actually fairly easy, as easy as OpenCore patches you do already. The hard part is flashing it & recovering from bad bricks, but if you got this down to science, it's all easy! At least we now know that some BIOS modding does work, and boots fine!

If you want to get started with BIOS modding, get the 1.80 UBU package here: https://winraid.level1techs.com/t/discussion-ubu-tool-related-questions-reports-and-suggestions/33251/1920

This second package is similar, but does not have all utils as the one above: https://winraid.level1techs.com/t/discussion-ubu-tool-related-questions-reports-and-suggestions/33251/1944

1.80 UBU package has everything, such as MM Aptio IV tool. See this guide how I inserted the NVME module, it's easy: https://winraid.level1techs.com/t/howto-get-full-nvme-support-for-all-systems-with-an-ami-uefi-bios/30901 ReBar module insertion is identical, use MM Aptio IV guide as well: https://github.com/xCuri0/ReBarUEFI/wiki/Adding-FFS-module

BUT! I think if the checksum for the BIOS changes, it'll stop booting. So need to make sure the header stays intact. See the binary diffs of my mods with the stock J396 in WinMerge, and you'll see what I mean.

bibikalka1 commented 3 weeks ago

@BillDH2k

Alright, I've patched 1 byte in meblast, can you check if it works as intended, keeping that byte at FF and triggering manufacturing mode? After you do this meblast, dump ME as in [fpt.exe -ME -d ME.BIN], then check that byte at the address 0x400. Btw, you can replace the BIOS version in the meblast tool with the latest J396, it'll put that one into ME. For Z820 meblast binary is the same, so we'd get 2 birds with 1 stone if this works! MEBLAST1.zip

As you are probably doing this already, unhook your main SATA drive before any testing, this way nothing you do can impact the bootable system. Once you put the good full BIOS image back, you'll be back to where you were.

For BIOS mods, I've done a few more versions: J61_0396_nvme_rebar_extra.zip J61_0396_nvme_rebar_mc_all_old1_extra.zip J61_0396_nvme_rebar_mc_all_new1_extra.zip

Maximalist mods are: J61_0396_nvme_rebar_mc_all_old1_extra.zip J61_0396_nvme_rebar_mc_all_new1_extra.zip

If these fail, try 1st this one: J61_0396_nvme_rebar_extra.zip

If the above fails, try these ones below: J61_0396_nvme_rebar_mc_all_old1.bin J61_0396_nvme_rebar_mc_all_new1.bin

Report back! Many thanks for testing this again! I have 1 single semi-production system, and sub-optimal programmer setup, so I do sweat when testing things on this system :)

"extra" mods are basically patches for ReBar mod, here is the patching log, and I did some manual tweaks afterwards to make it more surgical and have as little change in the BIOS as possible. _ PS C:\electronics\z620\UEFIPatch_0.28.0_win32> .\UEFIPatch.exe .\J61_0396_nvme_rebar.bin .\patches.txt -o J61_0396_nvme_rebar_stage1.bin parseFile: invalid data checksum 5Ah, should be AAh patch: replaced 11 bytes at offset 9584h B800000000100000004C3B -> B8FFFFFFFFFFFFFF004C3B patch: replaced 17 bytes at offset 4D52h B9FFFFFFFF490FAFC14903C0483BC1776C -> 6690669090490FAFC14903C06690906690 patch: replaced 17 bytes at offset 4E2Eh B9FFFFFFFF490FAFC14903C0483BC1776C -> 6690669090490FAFC14903C06690906690 Image patched PS C:\electronics\z620\UEFIPatch_0.28.0_win32> .\UEFIPatch.exe .\J61_0396_nvme_rebar_stage1.bin .\IvyUSB3.txt -o J61_0396_nvme_rebarstage2.bin parseFile: invalid data checksum 5Ah, should be AAh patch: replaced 13 bytes at offset EE34h DE10D301FFFF00000B00000014 -> 8680311EFFFF00000B00000010 Image patched

Previous, June 13th 2024: I've patched together 2 new versions with microcode updates manually. Very surgical. Please test, as before, on Z420 v1 that you already tested on, same procedure.

Old are fast microcodes, new are slow microcodes. Depending on your use case, choose. Try the one you don't want first, then the one you want. If the latter works - keep it for a bit.

If success, I'll move on to MEBLAST next, and I guess we could do Z820 BIOS patching if you want.

J61_0396_nvme_rebar_mc_all_old1.zip J61_0396_nvme_rebar_mc_all_new1.zip

BillDH2k commented 2 weeks ago

@bibikalka1

I've tested the "nvme" freature and it worked. Now Zx2 can boot directly off an NVME stick without the help of OpenCore!

Will test your new mods when I get back to work.

The Crisis recovery is actually simple and quick - took 2 minutes (restored to official BIOS). My hardware flash process involves a Raspery Pi, kind of slow and more labor. I have a qood quality clip (3M model) which makes the job much easier.

Regarding the Hackintosh patching by OpenCore, the ideal is not to permanently alter the original BIOS so the PC can continue function as it was for Windows or other OS (when mac OS is not used). So all patching are done on the fly to make the Intel PC appear like an genuine mac to the mac OS. New features missing from BIOS, such as NVME support, can be added too by OpenCore (but OpenCore needs to be boot up first, from an USB or other bootable device, like Sata drive). OpenCore is very flexible, and an amazing work, and it works for many different boards/BIOS.

Issue like USB3 support is a driver/hardware compatibility related. There is simply no macOS driver support for the TI USB3 chip (beyond Catalina). So I don't believe it can be fixed by BIOS mod, unless the mod emulates the chip as a compatible hardware that mac OS support. But I don't thick it worths the effort. USB3 functions can be easily added with a $15 compatible add on card.

Also thank you for tips/links for the BIOS mod technique. My main interest here was to add V2 support so get the last performance out of this old platform (also happy to gain the "nvme" features). As I stated earlier, I'd happy to test your new mods, as long as I can get around with my setup.

Best!

BillDH2k commented 2 weeks ago

@bibikalka1

Here is the test results (Note: limited to successful boot, no functionality test).

Test 3. MEBALST1 "MEBALST1 xxx.bin" returned the following "ErrorLevel = 1" message:

Selected photo Reboot did not enter "Manufacturing" mode. Checked the 0x400 byte at ME region, still "01".

Test 4. J61_0396_nvme_rebar_extra.zip Successful Post/Boot.

Test 5. J61_0396_nvme_rebar_mc_all_new1.zip Successful Post/Boot.

Test 6. J61_0396_nvme_rebar_mc_all_new1_extra.zip Successful Post/Boot.

Test 7. J61_0396_nvme_rebar_mc_all_old1.zip Post, but unable to boot USB (FreeDOS). Stuck with a blinking cursor at the top-left corner. During Post, can enter BIOS setup, but any changes made/save would not Post. Clear CMOS could POST again but could not boot USB.

Crisis BIOS Recovery did not work.

Test 8. J61_0396_nvme_rebar_mc_all_old1_extra.zip Same as Test 7. Can only recover with a hardware flash.

bibikalka1 commented 2 weeks ago

@BillDH2k

Appreciate your testing, and sorry for having to use the clip!

Just to double check, you did use MEBLAST1 on top of a fully working ME? I saw the same error, but only when running MEBLAST 2x in a row without hard reboot. I guess this is good data if you got that error starting for a working ME.

Can you check your CPU stepping on this specific machine that you are testing on? Run CPU-Z, save config, look for CPUID line: Name Intel Xeon E5 2660 Codename Sandy Bridge-EP/EX Specification Intel(R) Xeon(R) CPU E5-2660 0 @ 2.20GHz Package (platform ID) Socket 2011 LGA (0x0) CPUID **6.D.7**

Btw, BIOS has a feature to load BIOS, I think it's the 1st menu. Not sure if your computer could not see any USB ports at all, or just could not boot from a USB. I guess loading an official BIOS file from BIOS would have tested that. I wonder if a SATA drive would have booted. I wonder if crisis recovery did not work because it could not read USB ports! Did it blink at all?

Anyway, thinking out loud. Let me look at this a bit more, but if you have a bit of clarification regarding what I wrote above - please do clarify.

Many thanks again!!!

P.S. Btw, I have Z820 bios mods ready to go - but we still need a modded MEBLAST to load that efficiently on v2 Xeons. Z820 ME is identical to Z620, so all learnings should work fine.

Update: copy-pasted all microcodes from J391 to J396, please test if you have a chance J61_0396_nvme_rebar_extra_mc391.zip

BillDH2k commented 2 weeks ago

@bibikalka1

I redid the MEBALST1 test, as I likely lost track of the test conditions. This time, MEBLAST1 succeeded without error. I verified that 0x400 offset at ME region is "0x01". Next, reboot, no "Manufacturing" mode.

Now, the interesting part: I started over again. Right after executing "MEBALST1.com J61_0396.bin", I wrote "0xFF" over the 0x400-0x500 in the ME region, by "fpt -f dummy_FF.bin -A 0x5400 -L 0x100" (where dummy_FF.bin is just a binary file I created with 0xFF's of length 0x100 bytes). Reboot, FDO jumper still in ON (no change), no "manufacturing mode"!. Powered OFF, moved FDO jumper to OFF position, powered ON, -> "Manufacturing Mode"!

So, I had a different "Manufacturing mode", which I just realized now, only visible when FDO jumper is off. When FDO is enabled, "manufacturing mode" disappeared, and you loss full access to the BIOS (I tried to FPT a whole BIOS flash, and it failed with errors about # of regions of not erasable errors). So, this "ME" replacement trick is unlikely to work.

Regarding the CPU used for the above test (TEST 3 - TEST 8), I believe it is a single E5-1603, on a Z620 board (can confirm later).

The clarification on "not able to Boot USB (freeDOS)": I had a FreeDOS USB as my boot device, to run all the tests mentioned above. When running TEST 7/8, system POST (saw BIOS logo, etc), then stuck with a blinking cursor on top-left corner, no longer respond to any input (not even ALT-CTL-DEL). Had to force power off to recover.

bibikalka1 commented 2 weeks ago

Now, the interesting part: I started over again. Right after executing "MEBALST1.com J61_0396.bin", I wrote "0xFF" over the 0x400-0x500 in the ME region, by "fpt -f dummy_FF.bin -A 0x5400 -L 0x100" (where dummy_FF.bin is just a binary file I created with 0xFF's of length 0x100 bytes). Reboot, FDO jumper still in ON (no change), no "manufacturing mode"!. Powered OFF, moved FDO jumper to OFF position, powered ON, -> "Manufacturing Mode"!

So, I had a different "Manufacturing mode", which I just realized now, only visible when FDO jumper is off.

@BillDH2k

Wait, so you could run "fpt -f dummy_FF.bin -A 0x5400 -L 0x100" after MEBLAST1 and it wrote stuff? Then you moved the FDO jumper, rebooted, and got "Manufacturing mode" and full flash access with the FDO jumper off? When you did a programmer flash, you probably did not touch your FDO jumper at all ! Probably it does not matter then if it's MEBLAST1 or the original MEBLAST?

I mean, it seems your new way gets us full write flash access. Please confirm. If true - we are there.

P.S. FDO jumper disables ME, and let's one overwrite it. I guess when you put it back, it tries to run it, cannot, and triggers manufacturing mode.

BillDH2k commented 2 weeks ago

@bibikalka1

I will test that later.

But right, now I have another issue with my setup, which you may help me to correct - I encountered this error (see screen capture), corrupted GBE region? Selected photo (1)

Not sure how I ended up here. I did a few things in btw the tests, suddenly noticed this error (it might occurred earlier without my notice).

This is a Z620 board used for the above testing. I noticed from the BIOS System Information page, one of the NIC MAC address, had changed. There are two LAN ports: "NIC Controller (AMT) MAC" and "NIC Controller MAC". Normally, These two MACs are different by just "1" digit. Now, BIOS page showed: "NIC (AMT) MAC: XXXXXXXXF39A", "NIC MAC: XXXXXXXX0000". The "NIC (AMT)" is correct (matched my photo record), but the 2nd one is wrong, should end with "F39B".

Any idea how to fix this?

Thanks!

bibikalka1 commented 2 weeks ago

This is a Z620 board used for the above testing. I noticed from the BIOS System Information page, one of the NIC MAC address, had changed. There are two LAN ports: "NIC Controller (AMT) MAC" and "NIC Controller MAC". Normally, These two MACs are different by just "1" digit. Now, BIOS page showed: "NIC (AMT) MAC: XXXXXXXXF39A", "NIC MAC: XXXXXXXX0000". The "NIC (AMT)" is correct (matched my photo record), but the 2nd one is wrong, should end with "F39B".

@BillDH2k Well - an easy path is to reflash the full BIOS from good backup, probably all the way back to BB2011 era, but of course, make it BB2013. I mean - you want to revert to a known good state, where everything worked. You could probably diff GbE area, see what changed there too.

But instead of patchy fixes, just do the full re-flash, much quicker.

BillDH2k commented 2 weeks ago

@bibikalka1

I did that, a couple of times. Same error. It is possible, that I picked the wrong backup BIOS. Will try again.

bibikalka1 commented 2 weeks ago

@BillDH2k

I did that, a couple of times. Same error. It is possible, that I picked the wrong backup BIOS. Will try again.

If you sure you have the right backup, and it still does that - do the CMOS clear with the button. A couple of weeks ago I reflashed the full good bios, and it would not boot until I cleared CMOS.

bibikalka1 commented 2 weeks ago

@BillDH2k Install this package - UBU_v1_79_19d : https://winraid.level1techs.com/t/discussion-ubu-tool-related-questions-reports-and-suggestions/33251/1944

It has the latest UEFITool.exe - alpha 68. If you open your full BIOS dump there, you can inspect each region, it shows MAC address , ME version, etc.

BillDH2k commented 2 weeks ago

@bibikalka1

Did this test: 1) Start with a full working BIOS (this case: V393+ME8+BB2013. Powered on with FDO jumper ON, boot to FreeDOS USB. 2) "fpt -ME -f ME8_0396.bin", where ME8_0396.bin is the ME cropped from official 396 BIOS (note: 0x400 has "0xFF" value). 3) Power off. Move FDO jumper to OFF position. Powered ON, booted up, "Manufacturing Mode" msg displayed.

Now, tried to use FPT tool, encountered errors: "FPT -d save.bin" -> "Error 26: The host CPU does not have read access to the target flash area..." 'FPT -f BIOS.bin" -> "Error 25: The host CPU does not have write access ..."


C:\ZX20>fpt -d save.bin

Intel (R) Flash Programming Tool. Version: 8.1.60.1561
Copyright (c) 2007 - 2014, Intel Corporation. All rights reserved.

Platform: Intel(R) Patsburg Chipset - Reserved DID 0x1D41
Reading HSFSTS register... Flash Descriptor: Valid

--- Flash Devices Found ---                                                
W25Q128BV    ID:0xEF4018    Size: 16384KB (131072Kb)                       

Error 26: The host CPU does not have read access to the target flash area. To enable read access for this operation you must modify the descriptor settings to give host access to this region. To enable read access for this operation you must modify the descriptor settings to give host access to this region."

C:\ZX20>fpt -f BIOS.bin

Intel (R) Flash Programming Tool. Version: 8.1.60.1561 Copyright (c) 2007 - 2014, Intel Corporation. All rights reserved.

Platform: Intel(R) Patsburg Chipset - Reserved DID 0x1D41 Reading HSFSTS register... Flash Descriptor: Valid

--- Flash Devices Found ---
W25Q128BV    ID:0xEF4018    Size: 16384KB (131072Kb)

The PDR region exists in flash. Continuing will cause the loss of any pre-existing data. Do you wish to continue? Y/ or q to quit : y

Error 25: The host CPU does not have write access to the target flash area. To enable write access for this operation you must modify the descriptor settings to give host access to this region.


So, without FDO jumper enabled, you can't the BIOS region.

BillDH2k commented 2 weeks ago

If set FDO jumper ON, then we lost "Manufacturing Mode". FPT critical operation will fail.

C:>fpt -f z2x_bck.bin

Error 7: Hardware sequencing failed. Make sure that you have access to target flash area! Trying to erase the same block (iteration: 2)

Error 7: Hardware sequencing failed. Make sure that you have access to target flash area! Trying to erase the same block (iteration: 3)

Error 7: Hardware sequencing failed. Make sure that you have access to target flash area! Failed to erase this block 3 times!!


So we don't have the TRUE "manufacturing mode" (from your MEBLAST/V207 sequence).

bibikalka1 commented 2 weeks ago

So we don't have the TRUE "manufacturing mode" (from your MEBLAST/V207 sequence).

@BillDH2k OK - then I don't understand how you were able to recover your manually flashed ME with FF. I presume your did RPI programmer with FDO jumper off? Then you had manufacturing mode. I guess you put FDO jumper on, and it gave your write access to ME, but only ME? And that's how you recovered?

BillDH2k commented 2 weeks ago

@bibikalka1

By enable FDO jumper, I can write to ME region. By simply overwrite 0x400 byte with "0x01" allowed me to exit the "manufacturing mode", based on your discovery.

bibikalka1 commented 2 weeks ago

@BillDH2k

Can you try access region by region?Use my bat file. Try this script in manufacturing mode FDO off, and ME FF FDO on. You can give indices to the bat file such as "backup 01", etc, so each set of backups is different. fpt.exe -DESC -d %ffdo% fpt.exe -BIOS -d %fbios% fpt.exe -ME -d %fmeoo% fpt.exe -GBE -d %fgbeo% fpt.exe -PDR -d %fpdr% backup.zip

The idea is to figure out if we can write to the BIOS area only, with manufacturing mode but FDO off. BIOS area only requires this flag: [fpt.exe -BIOS ]

P.S. Your log says it was re-writing the BIOS including the early very protected Flash Description (-DESC), but it hit a snag when it go to the ME! If we can write everything but ME that's fine too. Then with the jumper we can return, and correct ME region only.

BillDH2k commented 2 weeks ago

@bibikalka1

"backup 0" produced the following: (regions missing meant not able to access)

FDOO0.BIN 4,096 BIOS0.BIN 11,468,800 GBEO0.BIN 8,192 PDRO0BIN 8,192 BIOV0BIN 458,752 BIOB0BIN 10,944,512 BBLK0BIN 65,536

Update: testing "fpt -BIOS -f "...

bibikalka1 commented 2 weeks ago

@BillDH2k

OK, missing ME and full 16MB image is OK. We can get ME later with the jumper :)

BillDH2k commented 2 weeks ago

@bibikalka1

Tried to write to the BIOS region:
"fpt -BIOS -f BIO_full.bin" and "fpt -f BIO_CROP.bin -A 0x580000 -L 0xA70000"

Both failed with this error: Error 7: Hardware sequencing failed. Make sure that you have access to target flash area!

bibikalka1 commented 2 weeks ago

@BillDH2k

OK - looks like ME FF / manufacturing mode trick does not allow BIOS area writing. It did not write FDOO0.BIN either?

Let's me think about this a bit more. But in the meantime, could you try that BIOS I uploaded above using the standard MEBLAST/J207 sequence? J61_0396_nvme_rebar_extra_mc391.zip

BillDH2k commented 2 weeks ago

@bibikalka1

TEST 9: J61_0396_nvme_rebar_extra_mc391.zip
Tested on a Z420 + 1620 V1 CPU. Successful POST/Boot.

bibikalka1 commented 2 weeks ago

@BillDH2k

Alright, I read about flash descriptor region here: https://winraid.level1techs.com/t/guide-unlock-intel-flash-descriptor-read-write-access-permissions-for-spi-servicing/32449

Basically, if we could flash [fpt.exe -DESC -f fdo.bin], we could permanently unlock all regions for writing, and don't need manufacturing mode. Not sure if you tested this in your manufacturing mode, or with the FDO jumper on. You can simply grab the first 4k of the fresh bios file, it has the regions unlocked with all FF, and copy those 4k to DESC. If this works - it'd be a lot simpler for v2 chips to just unlock the regions via the DESC. Just do a diff of your BIOS dump, and a fresh BIOS from the update package. Our stock & clean BIOS file descriptors looks exactly like the picture from the link above:

flash_descriptor

P.S. Thanks for testing the new BIOS! Let me swap 1 microcode, and I should be done with everything I needed in there :)

BillDH2k commented 2 weeks ago

@bibikalka1

You can simply grab the first 4k of the fresh bios file, it has the regions unlocked with all FF, and copy those 4k to DESC. If this works

"fresh bios file" = j61_396.bin from HP?

bibikalka1 commented 2 weeks ago

@BillDH2k

Yep, from HP. I cropped, see the included fdo: fdo_unlock_j396.zip

You can have a different version at the bottom in the signature, but that does not matter. The key bytes are on line x60.

BillDH2k commented 2 weeks ago

@bibikalka1

Just tested that I can write to DESC region, with FDO jumper ON: "fpt -DESC -f fdoo0.bin" (fdoo0.bin was from earlier dump)

Will try the "blank" DESC file.

bibikalka1 commented 2 weeks ago

@BillDH2k

fpt can be confusing, if the region is the same as the file you are writing - it won't write anything, so one has to squint to see that in the log! Trying a different binary so it would actually write should be a proper test.

Also, please find attached another iteration of the microcodes. Can work as the one today, or trouble like yesterday. That'd be the fastest version for this chip, if one suppresses microcode updates in Windows. J61_0396_nvme_rebar_extra_mc391_plus.zip

If this BIOS works properly, take it for a spin, try to go to BIOS, boot, etc. See that it's OK at the basic level.

BillDH2k commented 2 weeks ago

@bibikalka1

With FDO jumper ON, overwrote to DESC region with fdo_unlock_j396.zip.

Before rebooting, "fpt -DESC -d fd_tmp0.bin" (FD_TMP0.zip). Worm reboot: "fpt -DESC -d fd_tmp.bin" (FD_TMP.zip).

No "manufacturing Mode", after warm reboot. But "backup.bat" was able to read all regions.

Move FDO jumper OFF, cold reboot - "Manufacturing Mode"! "backup.bat" was able to read all regions.

Try to flash 0x58000 BIOS region, "fpt -A 0x580000 -L 0xA70000", in both cases failed, with "Error 7".

So, this appeared to be the same quasi-"manufacturing mode", like the ME FF trick.

bibikalka1 commented 2 weeks ago

Move FDO jumper OFF, cold reboot - "Manufacturing Mode"! "backup.bat" was able to read all regions.

Try to flash 0x58000 BIOS region, "fpt -A 0x580000 -L 0xA70000", in both cases failed, with "Error 7".

So, this appeared to be the same quasi-"manufacturing mode", like the ME FF trick.

@BillDH2k

OK, so it's similar and a little different, since now you can read ME with and without the FDO jumper. Before you could not read ME without the jumper.

I wonder if you should also put FF into ME into that position, so it would not run. So, basically, 1) blank FDO 2) ME with FF at x400; or, do proper fresh ME with 01, and do a warm reboot with the FDO jumper on

Let me read more why not everything can be written with the FDO unlocked.

BillDH2k commented 2 weeks ago

@bibikalka1

After cold reboot, with FDO jumper OFF, now "Manufacturing mode" msg is gone. BIOS showed that "ME disabled". May be this is why we are able to read all regions.

I tried to flash the saved DESC region back, did CMOS clear, system remained as "ME disabled". Again, no "Manufacturing mode".

bibikalka1 commented 2 weeks ago

After cold reboot, with FDO jumper OFF, now "Manufacturing mode" msg is gone. BIOS showed that "ME disabled". May be this is why we are able to read all regions.

I tried to flash the saved DESC region back, did CMOS clear, system remained as "ME disabled". Again, no "Manufacturing mode".

What does ME have now? You can probably move the jumper, and reflash it to its init state with 01. I guess your DESC region is now normal?

The tool says the regions should be open to read/write with blank DESC, I wonder if moving that password jumper to BB protection jumper is needed in addition to FD jumper. See the right panel in the screenshots below:

fd_clean fd_mine

bibikalka1 commented 2 weeks ago

@BillDH2k

Alright, it looks like we are hitting "Protected Range Registers". Its BIOS itself preventing us from writing to it. So do try to move that green password jumper to the BB protection pins, near the FDO jumper, and see if that helps.

Quote: Notice that BIOS/UEFI is not locked by the Flash Descriptor and thus OEMs are responsible for implementing BIOS-specific read/write restrictions such as Protected Range Registers and so on. That means that the Flash Descriptor is not responsible for any read/write restrictions set at the BIOS/UEFI region of the SPI chip, so the FD-related methods below will not be of any assistance in such cases.

BillDH2k commented 2 weeks ago

I had the green jumper moved from "Password Recovery" (next to AUX power connector) to E14 (BB), all the times.

So, this requires more investigation. Right now, I will try to recover by flashing working BIOS back (out of "ME disabled" state.

bibikalka1 commented 2 weeks ago

@BillDH2k

Does MEBLAST work with FD jumper on? I think that should work for you.

BillDH2k commented 2 weeks ago

@bibikalka1 Thanks for the suggestion. Will try that tomorrow.

bibikalka1 commented 2 weeks ago

@BillDH2k

Good session today! Try that latest BIOS tomorrow!

I am reading up on Protected Range Registers, I mean, those can probably be treated in BIOS, but that needs BIOS flashing functionality :)

P.S. I am thinking - that ME8/J207 combo must somehow unlock those Protected Range Registers, which are covered by some variables in that 0x510000-0x580000 area.

PPS One more thought - can bios writing in manufacturing mode only be available in J207? Perhaps before your restore ME, try to downgrade to J207 and see what it does.

BillDH2k commented 2 weeks ago

@bibikalka1

TEST 10: J61_0396_nvme_rebar_extra_mc391_plus.zip Successful POST/Boot.

This was done after recover from the "ME disabled" state, by flashing back a saved BIOS bin (MEBLAST/V207 sequence still working).

PPS One more thought - can bios writing in manufacturing mode only be available in J207? Perhaps before your restore ME, try to downgrade to J207 and see what it does.

Forgot to try this. So far, all we had tried indicating that without FDO jumber ON and "Manufacturing Mode" at the same time, full BIOS writing is not possible

bibikalka1 commented 2 weeks ago

Forgot to try this. So far, all we had tried indicating that without FDO jumber ON and "Manufacturing Mode" at the same time, full BIOS writing is not possible

@BillDH2k

I was reading about this now. I am thinking that they could have writable bios region in manufacturing mode back in j207, but closed this loophole later. Intel bios security and all that! So we could try j207 on v2, it has no microcode for those chips, but bootblock is compatible. Will probably spit out a missing microcode message :) I guess could test if j207 manufacturing mode always has writable bios, and then try something like j350 later to get v2 functionality. Anyway, an open question still!

Thanks for trying the latest bios! I am looking into unlocking some hidden bios menus now.

bibikalka1 commented 2 weeks ago

@BillDH2k

Found this, it's Lenovo, but identical issues: https://winraid.level1techs.com/t/lenovo-d30-thinkstation-c602-chipset-me7-sandy-bridge-to-me8-ivy-bridge-support/33917

Need to explore this afudos utility!

BillDH2k commented 2 weeks ago

@bibikalka1

Will try AFUDOS when I get around. https://www.bios-mods.com/forum/showthread.php?tid=1950

bibikalka1 commented 2 weeks ago

@bibikalka1

Will try AFUDOS when I get around. https://www.bios-mods.com/forum/showthread.php?tid=1950

@BillDH2k

So I did try a bunch of AFUDOS versions I found back in March 2024, before I discovered the ME/J207 hack. I kind of recall it did not work - but all these AFUDOS versions are sitting on my drive, so I did download them :) If I read the thread about Thinkstation carefully, it seems he started with effectively flashing that clean multi-FF FDO block you tried yesterday before proceeding with AFUDOS. So you may have to do that first, to try to follow the workflow for Thinkstation. I am uploading those for you here:

AMIBIOS_and_Aptio_AMI_Firmware_Update_Utility_2014.09.zip afudos_collection.zip

We still have no easy path to put a modded BIOS onto a v2 workstation, other than the programmer :( If AFUDOS does not work with J396, it may still possible it'd work with something older v2-compatible like J350 or J365.

I read about unlocking menus, it seems to be quite a lot of effort. Perhaps I better change a few hidden bios variables with that other UEFI tool, like this guide here: https://github.com/xCuri0/ReBarUEFI/wiki/Enabling-hidden-4G-decoding See the text file on the variables that exist in this BIOS: ZX20_menus.zip

Btw, I ordered a v2 cpu on eBay, E5-2643 V2 3.50GHz for like $12. That's crazy how cheap that is! Now I have E5-2660 0 @ 2.20GHz. But, I guess I'll flash the last modded BIOS plus ME8, will swap the cpu to v2, and I am locked for the time being.

I will put all Z620 mods into the Z820 BIOS, for completeness of the exercise.

BillDH2k commented 2 weeks ago

@bibikalka1

A V2 enabled Zx2 is still a very capable machine! As for the BIOS upgrade/mod, having a V1 CPU at hand + "ME8/V207" trick is still very good option, easier than hardware flashing, in my opinion.

BTW, I will find a time to try the AFUDOS and thanks for the links/suggestions!

bibikalka1 commented 1 week ago

As for the BIOS upgrade/mod, having a V1 CPU at hand + "ME8/V207" trick is still very good option, easier than hardware flashing, in my opinion.

@BillDH2k

Interesting thought on simply using a v1 cpu to do the modded BIOS flash, I did find the clip route very challenging a few weeks back. Limited physical room if it's in the case. Btw, do you remove anything when you do a Raspberry Pi clip? I mean removing cards, unhooking power supply from the motherboard, etc? Or is it all just clip on, and go? I had to remove the video card since it's so bulky, and simply blocked my clipping action.

Alright, i took the modded BIOS plunge! Sweaty shaking hands while moving the jumpers around, etc :))) Reminded me I am definitely not a hardware guy! I get these short time windows to mess up with the hardware, so always in a rush. And I hate the thought of having to recover a brick given no time! Anyway, thank you AGAIN for doing tremendous leg work testing everything and dealing with recovery methods and such!!!

I did meblast (with new J396/latest ME8), let is initialize itself normally. Backed up everything, did MEBLAST/J207. Rebooted to Manufacturing mode, flashed GBE/BIOV/ME from backup, and my crop of "j391_plus" version to the BIOB code area. Rebooting back to DOS was OK. I removed the SSD beforehand, put it back now, it gave me a flashing cursor. Did button BIOS reset, that seems to have fixed the SSD boot.

Renamed mcupdate_GenuineIntel.dll to *.dll.bak (after fixing the ownership issue), rebooted, verified that the old 2014 microcode was now in use. No difference in CPUMark that I can see. But, the system feels more snappy - placebo? This registry key had all microcode info disappear after I renamed mcupdate_GenuineIntel.dll : (HKEY_LOCAL_MACHINE\HARDWARE\DESCRIPTION\System\CentralProcessor\0) Need to read more about microcodes, seems things have evolved quite a bit since 2018.

Next, tested ReBar. Disabled Legacy support in BIOS. But! Found a weird UEFI variable "Launch CSM", in addition to another one, "CSM Support". booted to UEFI shell using this guide: https://github.com/xCuri0/ReBarUEFI/wiki/Enabling-hidden-4G-decoding

0x95982         One Of: Launch CSM, VarStoreInfo (VarOffset/VarName): 0x17F, VarStore: 0x1, 
0x959A8             Default: DefaultId: 0x0, Value (8 bit): 0x1 {5B 0D 00 00 00 01 00 00 00 00 00 00 00}
0x959B5             One Of Option: Enabled, Value (8 bit): 0x1 (default MFG) {09 0E 03 00 20 00 01 00 00 00 00 00 00 00}
0x959C3             One Of Option: Disabled, Value (8 bit): 0x0 {09 0E 04 00 00 00 00 00 00 00 00 00 00 00}

This variable "Launch CSM" was found in 2 places, one had value of 1, the other had 0. This is a bit buggy, but very easy to correct. Set it to 0. All good.

Rebooted to Windows, now all ReBar attributes are YES in GPU-Z except for the videocard hardware, enabled ReBar using the small utility. Rebooted again - ReBar is fully supported, I get full 8GB on my ancient RX580. Enabled Catalyst to use it by adding registry keys as per instructions elsewhere to [HKEY_LOCAL_MACHINE\SYSTEM\ControlSet001\Control\Class{4d36e968-e325-11ce-bfc1-08002be10318}\0001] At first was adding keys to 0000, but no go, opened RegEdit, and I saw it was actually 0001!

Now Catalyst has a toggle for ReBar, and it's ON! So Rebar is in full action, both in PCIE space, and on the legacy video card.

Will probably test NVME boot soon, and once everything is good - will swap the cpu to v2. Overall, I feel I am done with the cpu modding scope - no more good ideas that are really needed.

On AFUDOS, I am attempting to reach that @nikey22 guy to find out which AFUDOS he used, and if that's of use to us. Btw, for AFUDOS flashing tests, I suggest that you downgrade to something like J365 first. I see J61_0365.BIN has the date of Dec 2013, and some of AFUDOS versions are from 2014, so it may help to have AFUDOS be newer than the BIOS file.

P.S. So I was thinking that it would also be interesting to test j207 & j365 with dummy ME that has FF at that specific address. The question is if j207 would provide full flash write access in that scenario as well, similar to the current meblast/j207 method. Then j365 with dummy ME would test if the full write access is still there, to enable v2 cpu mod flashing.

BillDH2k commented 1 week ago

@bibikalka1 Had been busy/away and just came back to this.

For RPI flashing, I found out that I didn't need to remove power supply. I used a Samsung 5V charger, and 3M clip.

signal-2024-06-28-140639

signal-2024-06-28-140653

Thanks for your great work. I will try to test AFUDOS route when I get around.

bibikalka1 commented 1 week ago

@BillDH2k

Your power supply is not even that beefy! There are 6 or 7 holes next to the bios chip, I measured 3.2 volts off my mediocre power supply. I guess it's a forgiving electrical setup!

Summer is slow, nobody is responding to me on the raid forum :) I am still confused about older vs newer microcodes, and if they influence performance.

Thanks for testing stuff - no rush, whenever you find a reasonable chunk of time. J365 in manufacturing mode is most interesting, it might just enable full flash access just like j207!

BillDH2k commented 1 week ago

@bibikalka1 3.2V is not bad, it is only a small drop from 3.3V off the RPI board. The 5V powers the RPI board, which in turn provides 3.3V used to flash the BIOS chip.

bibikalka1 commented 1 week ago

@bibikalka1 3.2V is not bad, it is only a small drop from 3.3V off the RPI board. The 5V powers the RPI board, which in turn provides 3.3V used to flash the BIOS chip.

@BillDH2k

Right! For a while I just could not talk to the flash chip - so I started chasing various things, including voltage. 3.2V did look good, and a better clip worked fine!

So today I put that v2 Xeon I got recently into the system. Cleaned up the fan from 10+ years worth of dust! So far everything seems to be working correctly. Of course, I can no longer flash the modded BIOS, that is until a new method is found! :)

My board was super old, it had 1.02 BIOS from the factory in it, if I read the DESC signature. So I guess the original guide was right - there were no electrical differences in the v1/v2 boards, and all the difference was in firmware.

bibikalka1 commented 2 days ago

@BillDH2k

I am now thinking of sticking an overclockable v2 Xeon into my Z620, and trying to crank it up to get great single threaded performance. I guess with the older microcodes it's all set up to be as if it was year 2014! https://h30434.www3.hp.com/t5/Business-PCs-Workstations-and-Point-of-Sale-Systems/Z620-Z420-v2-overclocking-best-practices/td-p/9113778

Is there anything interesting on your side? I am waiting to put my guide together on modded BIOS until the v2 Xeon full BIOS write access is tested, and either works, or doesn't. Technically, one could probably remove write protection from the modded BIOS such that a modded BIOS would allow to be reflashed to another modded BIOS. But this is a fairly non-standard mod only required for brand name workstations, so I cannot draw on much guidance out there.