search

zearp / OptiHack

Dell OptiPlex 7020/9020 Hackintosh Stuff
https://zearp.github.io/OptiHack/
155 stars 53 forks source link

Monterey public beta enroller error #57

Closed mgrimace closed 3 years ago

mgrimace commented 3 years ago

Ok, I'm having an error when attempting to enrol in the public beta using the macospublicbetaaccessutility. After running it, it fails on the validation step and gives this error:

Screen Shot 2021-07-21 at 7 42 42 PM

I'm on Catalina, and updated my serials from iMac15,1 to Macmini7,1 using genSMBIOS. I also set securebootmodel to disabled. I used my existing working EFI and just switched up the serials and usb map kext (being sure to replace the SMBIOS in the .plist there too). So my first step to troubleshoot was to grab the latest stable EFI. I replaced serials, etc. Now, using the stable EFI, when I boot it doesn't even show Macintosh HD as an option in the OC picker. I had to boot from my USB key backup EFI.

IMG_0533

I don't even know what's going on there, so I reverted to my working EFI with the new Macmini7,1 serials. iCloud is working with the serials and everything else seems fine otherwise. It seems like something to do with the newest EFI isn't working, but really the only thing I've changed is the serials. The main difference in EFIs that I can tell is that there's some additional values in the new 'generic' section where I input serials vs., my older EFI.

Note the configs are not actually zipped, I just had to add the extension so I could upload here, just remove .zip from the end. configDOESNTSHOWHD.plist.zip configWORKING.plist.zip

So issues:

  1. Can't enrol in beta
  2. Newest EFI being weird
  3. Related? Maybe I screwed something up with the serials

I'll attach broken and working configs to this post (serials removed, except I wonder if those are the issue). I apologize in advance if this is something obvious I'm missing.

zearp commented 3 years ago

I have never used the enrolling app, it should have a log if you wanna know what happened. In the top menu you can have it show the log while running, like in the full installer. You can enrool using a command too: /System/Library/PrivateFrameworks/Seeding.framework/Versions/A/Resources/seedutil enroll DeveloperSeed

I'll see what's going on with the EFI... installed beta 3 a few days ago on a fresh usb stick and worked there... maybe something broke somewhere.

mgrimace commented 3 years ago

I have never used the enrolled, you can get to the same with a command: /System/Library/PrivateFrameworks/Seeding.framework/Versions/A/Resources/seedutil enroll DeveloperSeed

I'll see what's going on with the EFI... installed beta 3 a few days ago on a fresh usb stick and worked there... maybe something broke somewhere.

Command worked! Monterey showing up in upgrade menu!

No idea what's going on with the EFIs. As a test I generated some new serials, thinking maybe I screwed up and left an old one in or something but the new serials didn't make a difference either. I tried resetting NVRAM as well. I also tried both the latest stable build, as well as the most recent updates to the EFI (downloaded the current code as a zip, then used the EFI there with my serials). No honest idea

mgrimace commented 3 years ago

I should mention re. EFIs that was just to boot back into Catalina with the new serials, not even the installer (I hadn't gotten the beta enrol working by that point).

zearp commented 3 years ago

I've pushed a clean build which I just tested to work. Will now check the release to see what's going on in there. I noticed sometimes the files in BOOT don't get updated despite me also pushing changes. It happened once or twice and could cause an issue as it works on conjunction with OpenCore.efi usually it doesn't matter much. Lets see if I get the same issue. On your screenshot it doesn't show APFS partitions, looks like file system stuff is not properly loaded. A guess haha.

Brb.

zearp commented 3 years ago

When I boot with the release EFI it shows all partitions for me, including HFS ones. No clue why nothing shows for you.

Just downloaded the release and pasted in my serials and booted from it including one nvram reset for good measure.

Really no clue!

mgrimace commented 3 years ago

The Monterey installer also didn't work, error'd with a gatekeeper error (when installing from system update menu). I also tried disabling gatekeeper just in case, and it still error'd with some kind of nonsense % missing error. Clearly something's wrong on my end, going back to iMac15,1 serials and Catalina, then I'll maybe try Big Sur.

Screen Shot 2021-07-22 at 7 41 17 AM
zearp commented 3 years ago

I would make an offline installer instead. Seems updating from previous macOS version from within the OS isn't sorted properly. Lovely error though haha.

You can download this and when you install it will create the Install macOS Monterey app in /Applications for you.

Installing a newer macOS on top of an older one will work fine. A clean install and then restore your stuff from a Time Machine backup will also work fine. Only downgrading can be tricky, though restoring your stuff from a backup made with a newer macOS on an older one will also work.

mgrimace commented 3 years ago

I'll give the offline installer a try. I think my steps are

  1. Now that I'm back on Catalina, pull the newest EFI and use my working Catalina serials and see what the heck is going on there. I think the only thing I'd need to change in the config is the string with macmini7,1 to imac15,1 and otherwise it should work.
  2. Update into Big Sur using working serials (see if that update goes through)

  3. Gen some fresh Monterey serials, make sure the EFI boots, then update with the standalone installer

    I'll close this issue, it's a hot mess right now but I'll post some results/updates in here if I find anything.

mgrimace commented 3 years ago

Ok, newest EFI with literally nothing changed but serials, ROM, and Macmini7,1->iMac15,1 in the generic section. After booting into the new EFI it also does not detect my harddrive/partition with the OC picker (same issue as above). I didn't think it was the serials but it was worth ruling out.

So what this means: I stopped updating the EFI once the SMBIOS changed on the main branch (intending to stay on Catalina). Something that changed between "old" iMac15,1 EFI and "new" Macmini7,1 EFI has resulted in the EFI not being able to detect my hard drive on boot. I noticed a few new properties under generic but after a quick search nothing noteworthy that would cause this issue. Any suggestions where I should test next? Should I open another issue to track this or just leave it all here?

mgrimace commented 3 years ago

Seems like it could be an opencore issue, doing some searching: https://www.tonymacx86.com/threads/big-sur-os-x-drive-does-not-appear-in-the-opencore-boot-menu.302747/

mgrimace commented 3 years ago

Ok it looks like this is an open core issue, I'm still reading up on it, but that would make sense why the older EFI is working for me (older version of OC). I tried the first troubleshooting step of installing intel power gadget (which, during the install corrects the name/location of the boot drive). Didn't work for me but there's some manual steps.

zearp commented 3 years ago

That topic is not related I think. With Big Sur updates and maybe also in Monterey your boot entry can get a different name sometimes. I had the Preboot thing too, booting into it resulted in a normal boot to the desktop. You can name that partition back or ignore it. I think this was fixed in newer builds and maybe OpenCore did something about it too.

You experience something different cuz you have no partitions showing at all. No APFS or HFS , no NTFs, nothing at all.

I don't know how I can reproduce this, I reset my nvram and booted with the current and release EFI and could see partitions on the installer and on my internal disks. Really no idea what else I can try.

Did you at some point have the old Bootstrap stuff enabled, or maybe the newer LauncherOption? I think both or one of them loads itself somewhere persistent to always be able to boot OpenCore in case of another OS messing things up. It would be super rare but the only thing I can think of that could cause something like this.

Check the boot entries in the bios, remove them all and/or load bios defaults so it repopulates those on the next boot. If that also doesn't work maybe wiping cmos and all the other stuff using the combined Dell methods from the guide can help. That would clear all the nooks and crannies something might hide. If that also doesn't help then I really don't know what else it could be.

mgrimace commented 3 years ago

Very strange. So I do have partitions showing normally on my older efi (loaded to my internal drive), it's only when I use the newer EFI that the partition doesn't detect. That's what's making me think something to do with the newer version of opencore. Never touched anything Bootstrap or Launcher, basically I just pull the newest EFI, add my serials/ROMs, and add my usb port map kext, then reboot. Been doing that forever, but stopped with the new SMBIOS changes because I wanted to sort out 4k/dual monitors before doing anything else.

In terms of further troubleshooting, I reset the bios defaults, the only boot entry is the internal hdd UEFI. To solve the problem, I boot using the installer with the older EFI, then once I start up, I replace the internal drives EFI with the older one, reboot and Macintosh HD shows up back to normal in the picker from the internal drive.

As a precaution, I'm restoring from a time machine backup (back to Catalina), in case all my testing with configs for 4k and serials messed something up. Otherwise, I don't know what to do other than just leave it on the older EFI/OC version.

I also went to the Dortania troubleshooting guide (which is where I should have started) and found this: https://dortania.github.io/OpenCore-Install-Guide/troubleshooting/extended/opencore-issues.html#can-t-see-macos-partitions - I tried changing those values but nothing.

After the restore I'll see where things are at. It's an old hdd so it's going to take approximately forever.

zearp commented 3 years ago

I totally missed that your old EFI shows everything fine!

Hmm, still very weird it doesn't show with the new one. I don't know how many things changed but it would be interesting to drop your old config in the new EFI and see what happens, it would quickly rule out if a config change causes this or OpenCore/etc. Hopefully that aren't any critical errors and just warnings, we only need to get in the picker.

Another thing is to try with the BOOT folder of your working one in the new EFI.

There is a difference between old/new EFI that causes this, drivers, kexts, etc. I don't know how old your old one was so the list of differences may not be too long. You can post both full EFIs if you want me to check it out.

I don't understand why I can't get the same to happen though. We have identical machines and the only differences would be stuff stored in places we can purge. I tried that but still can't reproduce so it is really weird to me.

We default to 0 ScanPoliciy so it shows everything. It may not be the "best" for a final install but when setting up I want to see everything in the there.

mgrimace commented 3 years ago

Ah! I should have tried those first! If the restore doesn't fix things that'll be my next step:

  1. Test old config, new EFI
  2. Test old BOOT, new EFI

In the meantime while it's restoring, here's my "old"/working EFI. The "new" one is exactly the current posted EFI (as of July 22 1pm EST) with the only changes being my serials and Macmini7,1 replaced by iMac15,1, all in the generic section. I also copy over the "old" usbkext port into the "new" EFI kexts folder (so it wouldn't change). EFI-old iMac working anon.zip.

The same thing happens with both the old SMBIOS/serials on iMac15,1 and also when I tried the new SMBIOS/serials with Macmini7,1. With both SMBIOSes the "old" EFI boots fine and recognizes my internal HD when booting from the internal drive (no installer/USB present). The "new" EFI doesn't recognize my internal HD partition when botting from the internal drive, so I need to use my backup USB installer EFI to boot into MacintoshHD. Putting "old" back onto the internal EFI puts things back to normal/working internally.

zearp commented 3 years ago

It's a long shot but what happens when you set prev-lang:kbd back to string and en-US:0 cuz thats pretty much the biggest difference between the configs.

Another difference is IRQ patch that I made again but I don't think we even need those to be honest, HPET/IRQ. Framebuffer also differs but surely not the cause. The rest was mainly logging related like AppleDebug and ApplePanic.

The language thing somehow fixes a rare issue where some people just get a grey screen when installing. Setting it to an empty data field fixes t. Maybe we need to set it to data but still supply a layout even though the Dortania guide says empty is fine. Worth a try since it's one of the few "big" changes in the config, other than new OpenCore entries.

I've also updated your old working config to include all OC changes and updated the kexts, I wonder if that one works ok. If it does try setting the language thing to data and leave the field empty.

EFI.zip

mgrimace commented 3 years ago

Ok, restored from a time-machine backup. I restored to point before I started messing with serials and new configs. Catalina, SMBIOS iMac15,1.

NEW LEAD:

Screen Shot 2021-07-22 at 7 54 53 PM

I don't know what to do with that information but perhaps there's something there?

mgrimace commented 3 years ago

Reading more here as well: https://dortania.github.io/hackintosh/updates/2021/06/07/acidanthera-june.html Also here: https://www.insanelymac.com/forum/topic/348064-how-to-opencore-069-070-differences/ There's even a bit about display rotation!

Display rotation
ProtocolOverrides > added AppleEg2Info (Boolean): replaces the Apple EFI Graphics 2 protocol with a builtin version. This protocol allows newer EfiBoot versions (at least 10.15) to expose screen rotation to macOS. It can be False.
NVRAM > Add > 7C436110-AB2A-4BBB-A880-FE41995C9F82 > added ForceDisplayRotationInEFI (Number): defines display rotation. It can be 0 for no rotation or any of 90, 180, 270 for matching rotation in degrees. Only enabled if AppleEg2Info=True.
mgrimace commented 3 years ago

Fixed OCS error discovered above by changing value advisewindows ->advisefeatures in generic in my "old" working EFI based on readings above. Still testing missing partitions, unfortunately I don't think this is it since this is already fixed in the "new"

mgrimace commented 3 years ago

Summary

So far nothing is making 'new' EFI detect my Macintosh HD partition. I'll keep at it and see what I can find!

Edit: One thing I wasn't sure how to disable/remove was the IRQ patch you mentioned, how do I disable that? I can also say for certain it's not framebuffers because I've been changing those around all over the place for testing 4k and that never created this issue.

zearp commented 3 years ago

Interesting to try out rotation stuff.

I'm not sure how useful those compare tools are cuz they can list a lot of optional fields depending on the sample/source config it uses. I use the ocvalidator tool that comes with OpenCore which will warn about missing or extra entries very strictly. I also tried that compare app but it was easier to just add your config to my work folder and check what would change on Github. There were very few actual differences and none that would have anything to do with what boot entries are displayed. Maybe only the language thing that somehow can fix a grey screen.

Those advice things are related to the boot protect/bootstrap stuff. I never enabled that or used it. They are work arounds for multi booting that are not really good to use, cuz it wil still apply lots of OC things to whatever OS you boot from OC. I would only boot macOS or Windows Bootcamp and maybe a Live Linux distro. Anything else I do with bios boot menu.

Since replacing the boot folder and config didn't work we can conclude the problem doesn't lie there. That just leaves either kexts (unlikely as they're not even loaded at that point), drivers and the OC binary itself.

I'm not convinced this an OpenCore bug but thats only based on me not being able to reproduce it. I would suggest to setup debug logging and boot with OpenCore debug version and see what it does when its scanning for partitions. That would be the best way I think, at least something I would do before opening a bug report. They will also need steps to reproduce or at the very least logs.

The basic debug OC log is pretty nice and will show exactly what it doing to great detail. It is also easy to setup. You can grab the latest OC builds form Dortania';s build section and you'd only need OpenCore for this and change the config to actually log them (Target to 67 I think).

https://dortania.github.io/OpenCore-Install-Guide/troubleshooting/debug.html

I'm bummed out I can't reproduce it, I don't know what else to try. I've wiped nvram, and cmos/etc.

Let me know what the OpenCore debug log shows.

zearp commented 3 years ago

The extra weird thing is that no partition is showing at all. So it's not something like HfsPlus.efi being broken, it's everything relating to file systems. Hopefully the log will shine a light on it. APFS support is builtin so I would expect you can see APFS partitions but they also were missing in your picture.

mgrimace commented 3 years ago

Well. This is.... something.

Newest EFI, my serials, my usbkext, no change to lang. Setup Debug loggin by downloading the Debug version of OC. Followed the directions above and replaced BOOTx64.efi, OpenRuntime.efi, and OpenCore.efi with the debug versions, and added the Target=67 in config.plist (the other values were already set to true).

IT DETECTED MY HD AND BOOTED FINE.

So, I'm thinking it's got to be one of the .efi files I replaced, because I changed nothing else and now it works. I'm going to grab a fresh copy of OC and replace those just in case something... what... corrupted I don't know.

Here's the debug log in the meantime! opencore-2021-07-23-010402.txt

mgrimace commented 3 years ago

OK! Success.

Went back to new EFI, replaced my serials and my usbkext. Then I downloaded the standard release of OC 0.71 and replaced BOOTx64.efi, OpenRuntime.efi, and OpenCore.efi. As a result, it detects my MacintoshHD and boots into it just fine. I did notice a brief OCS error again, so I'll look into that.

So it's one of those 3 .efis, though I have no earthly idea why it wouldn't work when I copied the optihack release. I tried downloading the stable release, downloading the recent updates, and pulling the recent updates to my own branch and downloading them using GitHub desktop. I can't imagine how something would get messed up in that process but who knows. I'm tired and making mistakes so who knows.

I know you've been doing a ton of support for folks on here lately and I want to say thank you for talking this through with me and thank you for all your help. It's over and above. I hope that if this happens to anyone else at least we have a solution now and saves some other folks some frustration. I'll keep looking into the OCS error and still try and get to the bottom of things.

In the meantime perhaps dump a fresh copy of those .efis into the release just to be safe? Again, it's more likely something on my end, but I'm sure it can't hurt!

mgrimace commented 3 years ago

FYI, the OCS error on the newest EFI is:

OCS: no schema for GraphicsInputMirroring at 2 index, context <AppleInput>!

This appears for a split-second before the picker appears on boot/reboot (after the "Dell" logo, it's a flash and I have to use slow-mo on my phone to catch them). The OCS error that appeared in my older config was a pretty easy fix, I'll check into this one.

zearp commented 3 years ago

Not many options left, I think it's Opencore itself as APFS which is builtin (no driver needed) also wasn't showing, if it was HfsPlus.efi you'd still see APFS partitions. Only older macOS and installers use HFS+ still. Though you can run macOS on HFS+ you just won't get any updates. Apple made those dependant on APFS since Mojave or Catalina.

I tried 2 fresh builds yesterday, one from pre-compiled releases and one compiled myself from scratch (eg wipe everything). Sometimes it doesn't build the files in BOOT but the other files went fine. It's easy to tell on Github as it shows days since last change. I noticed it a while ago that sometimes files in BOOT were very old despite them supposed to have changed. Not sure what causes that but I don't think thats the cause here, and they're uptodate for sure now. If something in the current or release EFI makes no partitions show then I would expect me and/or others to run into this too.

Don't worry about no schema errors, its just a warning of missing entries. Nothing to worry about, easy to add the missing GraphicsInputMirroring entry and not crucial at all, anything crucial it will refuse to boot.

It's here: https://github.com/zearp/OptiHack/blob/master/EFI/OC/config.plist#L736-L737

Pretty much all of the newly added things to OpenCore we use the default value so if not defined it should still be fine. None of the new options/fixes/quirks added apply to Haswell or the EFI. I keep always try to stay on top of these new config entries and add them as soon as I find out about them. The missing one for you I did enabled cuz the docs said it should be enabled on all h/w unless it causes issues.

mgrimace commented 3 years ago

If something in the current or release EFI makes no partitions show then I would expect me and/or others to run into this too.

Right? I have no honest idea why this happened. Maybe something left-over from all my recent testing with 4k where I've been swapping around new and old EFIs all over the place. I wonder if all my weirdness with Monterey earlier was a side effect of whatever this was (e.g., trying to use an older version of OC and issues with the new EFI/OC). Regardless, at least now it seems to be a local problem on my end, which is a relief that it isn't something in the EFI itself (e.g., for you/others).

I'm back to square one but now with a working, up-to-date EFI, so I'll try Monterey again once I tidy up all my backups and EFI folders so I don't make any silly mistakes. Volume is definitely APFS, I just mostly want smooth/reliable with working monitors so I don't tinker outside of framebuffers, etc. everything else is "stock".

If you do happen to get any reports of users being unable to see the internal partition in OC picker, the fix is fairly easy: user downloads a local copy of OC release and replaces BOOTx64.efi, OpenRuntime.efi, and OpenCore.efi (though likely just OpenCore.efi as you mentioned). It might be an edge-case, but who knows if others are tinkering around like I've been it may become an issue.

Thanks again for your help here!

Pretty much all of the newly added things to OpenCore we use the default value so if not defined it should still be fine. None of the new options/fixes/quirks added apply to Haswell or the EFI. I keep always try to stay on top of these new config entries and add them as soon as I find out about them.

Thanks also for keeping on top of all these changes, if nothing else I've learned a lot here and hopefully can contribute more!

Now that I've got this sorted, I'll turn my attention back to Monterey and testing 4k/dual (and possibly now a viable lead to fix rotation) once I get updated. Planning to do that this weekend/early next week.

zearp commented 3 years ago

I'm glad it is sorted!

Still thinking about what could've happened. My best explanation would be that at some point Bootprotect or on of those related features which have been renamed/removed/re-added in a different way was active and set stuff in nvram newer OC versions don't like but somehow the debug version didn't care for and booted fine.

OC can set nvram variables that won't be deleted when wiping nvram from OpenCore itself. Which would explain why wiping nvram didn't fix it cuz it didn't actually clear it fully. Not to say that nvram wipe would've fixed it but it is likely to me. In a way it's too bad the non-working version started working with a debug version as it would've been interesting to compare OC debug logs of it working and not working.

I don't know what version you were on, but maybe too many changes happened and somehow nvram got borked which got "fixed" by debug version that maybe ignores certain nvram entries or something. We can only guess at this point. The way to reproduce this might be by using the exact version you had and then try to enable/disable some of those options that could be removed by now and just see what happens. Not important though.

It does make me wanna look for another way to wipe nvram, maybe an additional tool can be added that purges everything in the nvram. Just for troubleshooting sake. I remember when I came from Clover I ran into so many super weird errors and behaviour and it was all due to Clover and OC sharing nvram and some vars causing issues in the other boot loader. A reproducible clean slate method is very important for troubleshooting. I couldn't get mine in the same state as yours despite us doing the same steps including nvram/cmos clearing. Yet I wasn't able to reproduce it so something must have been leftover somewhere. Which my guess is nvram as OC doesn't wipe it completely but protects certain things even if they cause issues. It can't know if they cause issues of course.

I think you will like Monterey, apart form a few Safari niggles it is very polished already and runs very smooth. And most importantly for me, they sorted the Preview app to be more like Catalina again. Gonna be interesting playing with those rotation options. Hopefully its possible to fix it in macOS itself, if not we could add entries to the config that are disabled by default that can be enabled and set certain screens rotation up. I will also check how the rotation is behaving here, I rarely look at that to be honest.

zearp commented 3 years ago

From the Dell website:

The information stored in the BIOS (System Setup), known as Extended System Configuration Data (ESCD) can occasionally become corrupt due to various reasons such as power events, incorrect settings, hardware incompatibility due to a specific setting, or a turn on self-test (POST) or video issues. In these cases, it is sometimes necessary to reset the BIOS or CMOS (Complementary metal–oxide–semiconductor) to factory default settings, or in other circumstances, clear the nonvolatile random access memory (NVRAM).

The Real-Time Clock Reset (RTCRST) jumper helps reset or clear the NVRAM on the computer. The ESCD information that is contained in the NVRAM can be cleared by following the steps that are mentioned below. The NVRAM is cleared when the jumper is set to the closed position and turning on the computer for 10 seconds.

I'm hoping maybe someone made a .efi tool for this we can add to the EFI, else I will add Dells steps to the guide.

zearp commented 3 years ago

There is a CleanNvram.efi bundled with OpenCore but the docs say its an alternative to the builtin functionality. It may do exactly the same. Will have to check the source and compare what they do and/or test it by manually adding nvram vars the built-in one won't touch when clearing.

mgrimace commented 3 years ago

I'll try! I'm not sure if OCdebug kicked it out of the error-state, or if something was actually wrong with those three .efis and replacing them fixed it. If it's those three .efis directly, then re-using the "broken" EFI should reproduce the issue. If I am able to reproduce it, how do I dump the contents of the nvram? If it's in terminal, would booting using the external USB's working EFI still allow me to grab the 'corrupted' nvram?

zearp commented 3 years ago

Do you remember what version you were on? After OC 0.6.4 a lot of things changed internally and my guess is that nvram vars that were protected by OC's nvram clear option were causing OC not to initialise itself properly. The debug version may have "bypassed" that or something.

The next time something like this happen I will be sure to ask the person to dump the content of the nvram too plus try the Dell method, or my combo method to wipe everything and see if that helps. I assumed clearing nvram from the OC menu would clear everything but it protect certain variables. Makes sense but also doesn't make sense if you want a clean slate.

It may have been even more complex but hard to find out now that things are back to normal. It makes more sense to me that something was messed up in one of those places settings are kept that aren't easily fully cleaned (UEFI/nvram/cmos/etc).

Like you said, you tried many things and it is possible for these storage spots to become corrupted and just cause very weird issues that make little sense.

mgrimace commented 3 years ago

! I can reproduce the error by using the new EFI without replacing BOOTx64.efi, OpenRuntime.efi, and OpenCore.efi. I'm not sure if I'm saying that in a way that makes sense:

Steps to reproduce error:

Workaround

Fix error

This would suggest to me that it's likely one of those actual .efi's that's the culprit. As soon as I replace them with the ones I downloaded myself it works.

Regardless, I created the error and used the workaround but not the fix yet so it still should be in NVRAM. How can I dump that log for you?

zearp commented 3 years ago

Hmmm.. very interesting! That would almost imply the SMBIOS might be at play too. I will try your steps and see if I can get the same to happen here.

Dumping the nvram can be done with nvram -p it prints everything in there if I'm not mistaken, some sensitive info might be withheld depending on OC config. But I think thats only relating to OC versions. I will check the docs.

ExposeSensitiveData
Type: plist integer
Failsafe: 0x6
Description: Sensitive data exposure bitmask (sum) to operating system.
• 0x01 — Expose the printable booter path as an UEFI variable.
• 0x02 — Expose the OpenCore version as an UEFI variable.
• 0x04 — Expose the OpenCore version in the OpenCore picker menu title. • 0x08 — Expose OEM information as a set of UEFI variables.

We have it set to the default of 0x6.

zearp commented 3 years ago

Haha, my nvram has a complete kernel panic in it! I'll try wiping nvram from OC and see if that still exists after doing so.

% nvram -p
preferred-count 3
boot-args   debug=0x100 keepsyms=1
aapl,panic-info %f0%b0;=F%8d%e1u%10%0c4%0e%b3%d9e9%08%867%9b%cdf%b3%f9f%0e%93%cb%b5qn<K%e9@%e1%f9%bc,%a7%b7%e7g%100%08%87%b3%cbI7%bd%cc%1e%be%ddt%f9%9b%cd.%cb]c8\%17%bb%cdn7%054%ecN%8f%d7e2%88%1e%9e%afA0%bc%d9l6%9b%cd8[%ae<%16%97m9%98%0e%14%a3%d9@t%b4%bc%1c&%cfu x%9a%0c%82%e9@%eb%b2%dc]f%7f%e9%e1%f9Z!%0c%8f%d7tyx\%06%a1%86%d0*%08%96b%81%e0awz%bc.%93At%b4%bc%1c&%eb@0%bc%d9l6%9b%cd8[XF%be%cd`0%18%0bd%94%87%dbe%90%0e$-%d3%ebr7(H&%cb%cb%f3%b9%02%867%9b%cdf%b3%99%0c%bb%91k%b3Y%ce%0c%03%e9@0%bc%d9l6%9b%cd8X%ccf%0b%e3fd2%a8%1d%1e%a3%bf%eb%b2%dc]f%83t%a0/:%ec&%b3%cb_rY\?%9f%cb%f2/]%1e%86%83V %18%9e%16#+`x%b3%d9l6%9b%c9%b0%1b%b96%9b%85i0%90%0e%04%c3%9b%cdf%b3%d9%8c%83%c5l8%18%0c6%ae%81%da%e11%fa%bb.%cb%dde6H%07%fa%ae%c9%f0oz%86%b3}%e9%f20% -- cut here cut goes on for a while --
mgrimace commented 3 years ago

Dumped the NVRAM, results below. Note that these results are after the error and after booting with the installers working EFI. I have not rebooted into the fixed version yet.

nvram -xp:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>LocationServicesEnabled</key>
    <data>
    AQ==
    </data>
    <key>SystemAudioVolume</key>
    <data>
    Jg==
    </data>
    <key>bluetoothActiveControllerInfo</key>
    <data>
    jYKsBQAAAABzFIhj34rUEw==
    </data>
    <key>bluetoothInternalControllerInfo</key>
    <data>
    jYKsBQAAcxSIY9+K1BM=
    </data>
    <key>boot-args</key>
    <string>-v</string>
    <key>csr-active-config</key>
    <data>
    AAAAAA==
    </data>
    <key>fmm-computer-name</key>
    <data>
    TWljaGFlbOKAmXMgaU1hYw==
    </data>
    <key>prev-lang:kbd</key>
    <data>
    AA==
    </data>
    <key>run-efi-updater</key>
    <data>
    Tm8A
    </data>
</dict>
</plist>

nvram -p

bluetoothInternalControllerInfo %8d%82%ac%05%00%00s%14%88c%df%8a%d4%13
prev-lang:kbd   %00
fmm-computer-name   Michael%e2%80%99s iMac
bluetoothActiveControllerInfo   %8d%82%ac%05%00%00%00%00s%14%88c%df%8a%d4%13
SystemAudioVolume   &
run-efi-updater No%00
LocationServicesEnabled %01
csr-active-config   %00%00%00%00
boot-args   -v
zearp commented 3 years ago

Your nvram vars look nice and clean, mine was a multi page mess but most of it is gone now after clearing from OC menu. I now just have some basic entries left like yours. I'll go do some work and then try your steps and hope I can get the same to happen. I find it funny somehow there was a complete panic in my nvram.

I have dedicated a test machine for this (and my Monterey testing), will pull the disk out and a clean disk and do a full nvram/cmos/etc wipe of everything using my own combo method and then try your steps. Then use a clean install or a clean install Catalina backup. I'll also use the iMac SMBIOS and Catalina and then change SMBIOS. I'll follow your steps as close as possible.

I also wonder if maybe changing the SMBIOS could trigger it, though I didn't run into any issues when testing out different SMBIOS for Monterey.

mgrimace commented 3 years ago

Hmmm.. very interesting! That would almost imply the SMBIOS might be at play too. I will try your steps and see if I can get the same to happen here.

Interestingly enough this also happened when I had the SMBIOS set to Macmini7,1. That's where I first noticed the missing internal partition error, but I ignored that at the time and just used my 'old' working EFI and changed serials there to try and press through thinking a fresh install of a newOS might just fix things. It's possible the Monterey errors I experienced above where because I was using the older OpenCore version. I'm not sure what version that was (I think 0.70). Monterey install errors are not relevant, just a curious linkage (and revealed this bigger issue).

Since I've conflated a few too many errors in this thread, re. no-internal error:

I'll have to retest with the Macmini7,1 smbios, there was a lot going on at the time, but I want to get everything installed properly first before I start messing with serials!

If there's any other info/tests that would help let me know, it looks like I can reliably reproduce the error, I'll also attach the unfixed/error EFI to see if that helps reproduce on your end (serials removed).

EFI (new, not working) anon.zip

zearp commented 3 years ago

Thanks for posting the broken EFI, on paper if I were to boot using that I shouldn't see any partitions. Will let you know what happens. I'm very curious but can't try yet.

zearp commented 3 years ago

Finally got to cloning my old Catalina backup from a spinning disk to ssd so I can use that to test instead of waiting for a clean install. But I think in hindsight a clean install might have been faster haha. It's amazing how long it takes to just clean up space so it can fit on my 60GB ssd. Xcode alone took at least 5 minutes just to delete itself (over 100k files in there). I forgot how slim Catalina can be when you remove all 3rd party crap, its less than 20GB now and could be smaller if you remove wallpapers, voices, etc. Big Sur and newer are Big Bloat compared to Catalina.

SuperDuper is reporting 18MB/s copy speed from the spinning disk, which isn't the OS disk. All the small files I guess. The disk itself can do about 80MB/s when writing big files to it but small files/random access is horrible on these drives. I don't understand how we put up with those disks for so long. Fine for large file backups (full images) but as an OS disk or cloned backups it is just too slow.

Went down to 12MB/s now, 5GB copied of about 20GB. I wonder how long it will take. Once it's done I can use the little 60GB ssd I'm cloning it to to test stuff easily. First thing will be just putting your broken EFI on there and then wipe all the nvram/etc and see what happens.

zearp commented 3 years ago

Of course I overcomplicated things, nothing new. I could've just tested the EFI on a FAT32 stick.

I've done both now and both times I can see FAT32 (EFI partition on other disks), APFS and HFS+ partitions. The only way to get the EFI you posted to show no partitions is to disconnect all drives. I didn't try NTFS or Linux partition types cuz it seems bit outside the scope here but I assume it would also show those.

Really baffled by this all. You'd think wiping all the things and booting from the EFI you posted would give me the same result but it doesn't. Specially the little FAT32 test is a good way to quickly test EFIs. Format an old stick as FAT32 and just put an EFI folder in there and you can boot it.

I even tried a few times; format stick FAT32, copy EFI you posted, boot into it and wipe nvram/etc then reboot again, every time it would show my internal partitions just fine. Also tried wiping everything using my combo method and then booting straight into the FAT32 still with the same result.

This is a real head scratcher and I know its solved, I'm one of those people who just wants to know but at some point have to call it quits too. I used a standard 7020 with no additional cards and also tried it quickly on my main 7020 with an Apple Airport card. I don't think it's hardware related all these boxes are pretty much the same. Same board, same bios, same everything. Unless you got a model without vPro (no sticker with a tiny 1 on the inside) our machines should be 100% identical.

I'll throw in the towel for this one for now. Really no clue what it could've been now that we know your nvram was ok. Unless it was really some corruption or something not shown with nvram -p. I assumed that command shows all the contents of it but it is an Apple util so I don't really know how it works, it may read everything and only print variables it knows and/or are set or compatible with it.

These kind of places where things get stored outside of the OS do interest me and I'd love to have a proper way to dump the contents of these things without putting an actual chip programmer on them, but maybe that is the only way to know. I might try some things some time in Linux also in relation to really clearing nvram. If I have way to dump the full contents of it then I can easily test what kind of things OC wipes and what it leaves behind. But those are not really relevant just my personal interest in how this kind of stuff works.

I don't really like it when an OS puts things it may depend on for proper functioning or even booting in places that aren't on the disk itself. I've seen lots of issues in the Linux world where UEFI stuff messes up Linux installs making them unable to boot. They just sit in a half working grub shell complaining. You have to resort to booting live distro's and installing all kinds of 3rd party tools to sort that UEFI mess out, even resetting bios doesn't always fix it cuz those vars are kept somewhere else on some workstations and servers.

tl;dr I give up, the problem is fixed and since I can't reproduce despite my best efforts I will have to create an x-file for this one 😅

mgrimace commented 3 years ago

That is absolutely baffling! The only difference between working and not working for me are the BOOTx64.efi, OpenRuntime.efi, and OpenCore.efi files. So to me, it would stand to reason that if one of those files was broken or corrupted in some capacity then it should reproduce the error on your end as well. But since it doesn't I have no honest idea! I'm going to go through and double check that I didn't mis-spell a serial/board UUID or something. That would be the only other thing that I can think of that wouldn't carry over to you since I removed the serials in the copy I posted.

Thanks for trying it so thoroughly on your end, even though it's fixed I do love to know 'why' as well (to prevent it in the future), but I don't think there is an answer here. Definitely an X-File!!

zearp commented 3 years ago

Yeah its really weird. I literally just unpacked the file you uploaded and booted with it. Without any disks attached it shows nothing, when I attach some usb disks and press escape it re-scans for drivers and finds/lists the partitions.

I was reading some documentation and was reminded of you mentioning the Apple logo changes size. Apparently thats due to HiDPI/4k screens and can be fixed by setting UIScale to 02.

mgrimace commented 3 years ago

Ok, I tried one other thing and enabled hidden files/folders on the internal EFI drive. Inside was a .trashes that had a whole bunch of old EFIs so I deleted those. There was also a .Spotlight-V100 folder which had an interesting .plist in it called VolumeConfiguration.plist. I kept a copy of that .plist just in case and deleted that whole hidden spotlight folder. There was also a folder called .fseventsd, no idea what that is, inside is fseventsd-uuid (unix executable). Deleted that too because why not at this stage. Rebooted with the broken EFI folder and everything else deleted.... nothing. Still no internal drives. I thought maybe the .trash EFIs were maybe the culprit but nothing!

edit: and to follow-up, no difference in serials, board, UUID, ROM, etc. between working/non-working.

Tl;dr thought of another potential lead/difference and enabled hidden folders in the internal EFI. Interesting, but deleting .trashes/spotlight/etc didn't fix the problem

zearp commented 3 years ago

I would hope both Dells UEFI implementation and OpenCore don't scan anything outside the EFI folder. Things like the trashes shouldn't interfere even if not emptied. It might be possible they got scanned regardless, which messed things up but would be really weird if OC did that. If anything it might have been Dell's UEFI implementation also very unlikely though. There are strict standards for these kind of things.

You could repeat my test and see what happens, use the broken EFI you uploaded here on a freshly FAT32 formatted usb stick and see what happens. There won't be any dot files/folders on there to mess thing up. You should in theory get the same results as me. The stuff inside Spotlight dot folders and files is macOS indexes for their search normal and expected if you have Spotlight turned on.

mgrimace commented 3 years ago

You could repeat my test and see what happens, use the broken EFI you uploaded here on a freshly FAT32 formatted usb stick and see what happens. There won't be any dot files/folders on there to mess thing up. You should in theory get the same results as me. The stuff inside Spotlight dot folders and files is macOS indexes for their search normal and expected if you have Spotlight turned on

Intriguing! I kept fixed EFI on my internal drive, and put broken EFI on a USB. Booted into the USB by holding F12 during dell and selecting the USB UEFI. NO INTERNAL DRIVES SHOWN WHEN BOOTING FROM USB using broken EFI!

mgrimace commented 3 years ago

Using the fixed EFI + fresh macmini7,1 serials the beta enroller software also now works no problem. Starting install of Monterey (trying the system update route first out of curiosity). Only change to working EFI was new serials, Macmini7,1, securebootmodel string Disabled, and replaced two instances of iMac15,1->Macmini7,1 in my usbkext plist. Everything seems to be working smoother this time around!

zearp commented 3 years ago

I'm glad things looking good so far! I had some issues with upgrades either within Monterey as to Monterey from Big Sur. Using the full installer app or an installer stick to install on top of existing the install went perfectly fine for me and also keeps all your apps and settings in tact. Essentially the same just a larger download.

When it comes to upgrading to a newer macOS you can always install it on top of an older one. The other way around can get tricky depending on the changes and differences. Still a clean install of an older macOS and using the migration tool should work for downgrades. It's pretty flexible compared to Windows.

You're using the old usb map if you have two entries for model, not a bad thing just FYI. I recently changed it. The old map was using AppleUSBMergeNub and the new one uses AppleUSBHostMergeProperties to communicate with macOS. Which is apparently a more reliable method for Big Sur and newer. Not what we had issues, this could prevent some. And it reduces the number of model entries from 2 to 1 haha.

mgrimace commented 3 years ago

I'm glad things looking good so far! I had some issues with upgrades either within Monterey as to Monterey from Big Sur. Using the full installer app or an installer stick to install on top of existing the install went perfectly fine for me and also keeps all your apps and settings in tact. Essentially the same just a larger download.

Yes you're right here, I'm creating a USB installer now, using the full installer from the applications folder crashed to the login screen instead of restarting. USB installer should work though!

You're using the old usb map if you have two entries for model, not a bad thing just FYI. I recently changed it. The old map was using AppleUSBMergeNub and the new one uses AppleUSBHostMergeProperties to communicate with macOS. Which is apparently a more reliable method for Big Sur and newer. Not what we had issues, this could prevent some. And it reduces the number of model entries from 2 to 1 haha.

Very old yes! Is there a way to copy over the data to the new file (e.g., from old to the .plist) rather than remapping?

zearp commented 3 years ago

You can use the map as is cuz you have no internal ports too, only the mini tower models have those. The only editing you may need is to set the integer of ports that have bluetooth on it to 255 so macOS sees it as internal.