search

dortania / OpenCore-Legacy-Patcher

Experience macOS just like before
https://dortania.github.io/OpenCore-Legacy-Patcher/
Other
12.38k stars 1.18k forks source link

Mac Pro 11.3 Beta NVME issues #110

Closed K-Hobert closed 3 years ago

K-Hobert commented 3 years ago

Describe the bug Im not sure this will fall within the scope of this patcher, Apple's changes in IONvmefamily has broken support with Mac Pros 5,1/4,1/3,1 the bug started with Beta 3 of 11.3 has continued to beta 4, this maybe the release version of IONVMefamily for 11.3 therefor breaking computability between NVMEs and Mac Pros.

To Reproduce Steps to reproduce the behavior:

  1. install 11.4 newest beta to a Sata or NVME drive
  2. Finish installation of Mac OS, restart and attempting to boot from an 11.3 drive with an NVME drive installed in a pci slot will cause a kernel panic

Hardware (please complete the following information):

Opencore cannot block the IONVMeFamily.kext

khronokernel commented 3 years ago

Well might as well have an issue open, I will however need a bit more info:

With those 2, should be a bit easier. Quite odd however though as no hackintosh I've heard has had NVMe issues so must be some odd ACPI or firmware logic of the machine.

Edit: Found a copy of Beta 3 (20E5196f), will still require a proper kernel panic log and beta 2:

K-Hobert commented 3 years ago

cannot find a beta 2 IONVMeFamily.kext, time machine doesn't back up Big Surs snapshots does it? Could you compare Mac OS 11.2.3 and beta 3

here's a kernel panic from Beta 3... 1615405891809

khronokernel commented 3 years ago

Interesting, would it be possible to maybe get an image right before the panic? Wondering if there's any extra logging info. Additionally io=0xff in boot-args will get you a lot more on IOKit's actions and nvme=3 should help with extra IONVMeFamily info

For things to mess with while I reverse the kext, try running NVMeFixup.kext. I recommend -liludbgall to get nice printing info from it

K-Hobert commented 3 years ago

IMG_7427 here's right before the panic without any debug flags ill turn debug flags on

K-Hobert commented 3 years ago

Heres Beta 4 panic debugs flags are switched on, from own main Mac Pro IMG_7481 verbose is moving to quickly to capture.. for more information I would need a dump, and I'm not so sure how id collect a boot kernel panic dump wasn't in logs folder

khronokernel commented 3 years ago

It’s missing symbols, don’t forget to add keepsyms=1

khronokernel commented 3 years ago

Also you seem to have forgotten to put a space between mdasd=1 and -liludbgall

K-Hobert commented 3 years ago

Correct the flags. Odd behavior it’s freezes most of the time at verbose but sometimes reaches an Apple screen with a bar , then crashes to verbose. Still can’t capture the Kernal panic information quick enough Seriously confused. it’s booted to Beta 4… NVME working

K-Hobert commented 3 years ago

With boot-args debug=0x144 io=0xff nvme=3 unfairgva=1 mbasd=1 -liludbgall -no_compat_check -v keepsyms=1 my Mac Pro will boot to beta 4 without any noticeable issue still underlining issue remains changing a debug flag will break boot. Removing -liludbgall disables Bluetooth and breaks other usb issues. a kernel panic happens without debug= without IO=0xff system halts at assert failed on ionvmefamily

khronokernel commented 3 years ago

Removing -liludbgall disables Bluetooth and breaks other usb issues.

I'm very confused how that would even happen, honestly sounds like a hardware failure and the race condition is showing. But assuming it works fine in other OSes, unsure

without IO=0xff system halts at assert failed on ionvmefamily

Interesting, so does look like a race condition regarding the NVMe controller probing. As if the hardware isn't ready when IONVMeFamily starts and so crashes

You could try adding nand-io-timeoutms=50000 to set the timeout to 50 seconds and remove io=0xff

Additionally, could you send an OpenCore DEBUG log? Wanting to see whether DisableLinkeditJettison is applying correctly

khronokernel commented 3 years ago

Seeing this comment, it honestly feels like a hardware issue more and more:

broken support with Mac Pros 5,1/4,1/3,1

You said MacPro3,1, 4,1 and 5,1, could you elaborate more on the hardware configuration used? If it's the same exact SSD across all machines, I have to ask you to try using a different model to ensure this isn't a localized issue

K-Hobert commented 3 years ago

Don't believe the Broadcom chip is failing has worked in normal use cases under linux, windows and more supported Mac OS hasn't been an issue. USB issue seems a normal usb issue with a 5,1 Mac Pro under Big Sur, unplugging a usb keyboard then reconnecting will cause a disconnect permanently until a reboot, and the bluetooth controller is connected to a usb port inside these Pro's

I'm using a pair of Samsung 970 evo plus firmware updated for Mac OS non-raid, in Sonnet's NVME controller. other PCI slots have a Titan Ridge, Sonnets USB 3.1 and a Vega Frontier.

For an opencore debug log do I just need to switch to the debug release?

khronokernel commented 3 years ago

Regarding hardware failure, was referring to the SSD itself. If there are no other users with this issue, I can't justify spending more time on what's most likely a local issue on your hardware.

If there's others with this issue do report, but unless there's a clear benefit to adapt the patcher I'm not all too interested in having the bug tracker be a support forum. Currently we're working on GPU acceleration and more time I spend away from that won't be very beneficial for the user base.

But as mentioned earlier, if multiple users can confirm this issue I will reopen this issue

K-Hobert commented 3 years ago

Thank you so much for the help! Really appreciated.

startergo commented 3 years ago

I had the same NVME kernel panic with OWC NVME card+970 pro. And up to 11.2 it was running fine. I will attach the kernel panic later.

khronokernel commented 3 years ago

@startergo What Mac Pro model? Also yeah the kernel panic log would be greatly appreciated

startergo commented 3 years ago

@startergo What Mac Pro model? Also yeah the kernel panic log would be greatly appreciated

It is a cMP5,1. I am still away from the machine for the panic log.

startergo commented 3 years ago

IMG_20210310_141534 This is the panic log

khronokernel commented 3 years ago

@startergo Thank you, could also test with io=0xff in boot-args to see if it fixes the issue? Currently my theory is this is a race condition on hardware probing, where for some reason the device isn't ready on boot. The arg did fix the issue for K-Hobert partially.

I hope to look more over the weekend again on exactly what part of the probe crashes, what's interesting is both of you have MacPro5,1's with Samsung drives. Wondering if this is something to do with Samsung hardware/firmware specifically

startergo commented 3 years ago

could also test with io=0xff in boot-args to see if it fixes the issue?

It trapped at the same location. Also then it was even worst, because I got endless scrolling text while booting through Catalina. I had to clear NVRAM to boot.

startergo commented 3 years ago

There is an AssertFailed just before the panic: image

startergo commented 3 years ago

Wondering if this is something to do with Samsung hardware/firmware specifically

Actually Big Sur is on: Media name : OWC Aura P12 480GB Media

startergo commented 3 years ago

Also the AppleNVMe Assert failed is not the reason for the kernel panic as it is present in Catalina's Bootlog_Kernel:

2021-03-27 07:15:10.67 kernel[0]: (IONVMeFamily) AppleNVMe Assert failed: ( 0 != data )
2021-03-27 07:15:10.68 kernel[0]: (IONVMeFamily) ReleaseIDNode
2021-03-27 07:15:10.68 kernel[0]: (IONVMeFamily) file: /AppleInternal/BuildRoot/Library/Caches/com.apple.xbs/Sources/IONVMeFamily/IONVMeFamily-470.100.17/IONVMeController.cpp
2021-03-27 07:15:10.69 kernel[0]: (IONVMeFamily) line: 5478
2021-03-27 07:15:10.69 kernel[0]: (IONVMeFamily)
2021-03-27 07:15:10.69 kernel[0]: (IONVMeFamily) virtual IOReturn IONVMeController::CreateSubmissionQueue(uint16_t, uint8_t)::2861:SQ index=0 entrysize=64
2021-03-27 07:15:10.69 kernel[0]: (IONVMeFamily) AppleNVMe Assert failed: ( 0 != data )
2021-03-27 07:15:10.69 kernel[0]: (IONVMeFamily) AppleNVMe Assert failed: ( 0 != data )
2021-03-27 07:15:10.69 kernel[0]: (IONVMeFamily) AppleNVMe Assert failed: ( 0 != data )
2021-03-27 07:15:10.69 kernel[0]: (IONVMeFamily) AppleNVMe Assert failed: ( 0 != data )
2021-03-27 07:15:10.70 kernel[0]: (IONVMeFamily) ReleaseIDNode
2021-03-27 07:15:10.70 kernel[0]: (IONVMeFamily) virtual IOReturn IONVMeController::CreateSubmissionQueue(uint16_t, uint8_t)::2861:SQ index=0 entrysize=64
2021-03-27 07:15:10.70 kernel[0]: (IONVMeFamily) virtual IOReturn IONVMeController::CreateSubmissionQueue(uint16_t, uint8_t)::2861:SQ index=1 entrysize=64
2021-03-27 07:15:10.70 kernel[0]: (IONVMeFamily) virtual IOReturn IONVMeController::CreateSubmissionQueue(uint16_t, uint8_t)::2861:SQ index=1 entrysize=64
2021-03-27 07:15:10.70 kernel[0]: (IONVMeFamily) ReleaseIDNode
2021-03-27 07:15:10.70 kernel[0]: (IONVMeFamily) file: /AppleInternal/BuildRoot/Library/Caches/com.apple.xbs/Sources/IONVMeFamily/IONVMeFamily-470.100.17/IONVMeController.cpp
2021-03-27 07:15:10.70 kernel[0]: (IONVMeFamily) file: /AppleInternal/BuildRoot/Library/Caches/com.apple.xbs/Sources/IONVMeFamily/IONVMeFamily-470.100.17/IONVMeController.cpp
2021-03-27 07:15:10.70 kernel[0]: (IONVMeFamily) line: 5478
2021-03-27 07:15:10.70 kernel[0]: (IONVMeFamily) ReleaseIDNode
2021-03-27 07:15:10.70 kernel[0]: (IONVMeFamily) ReleaseIDNode
2021-03-27 07:15:10.70 kernel[0]: (IONVMeFamily) file: /AppleInternal/BuildRoot/Library/Caches/com.apple.xbs/Sources/IONVMeFamily/IONVMeFamily-470.100.17/IONVMeController.cpp
2021-03-27 07:15:10.70 kernel[0]: (IONVMeFamily) file: /AppleInternal/BuildRoot/Library/Caches/com.apple.xbs/Sources/IONVMeFamily/IONVMeFamily-470.100.17/IONVMeController.cpp
2021-03-27 07:15:10.70 kernel[0]: (IONVMeFamily) line: 5478
2021-03-27 07:15:10.70 kernel[0]: (IONVMeFamily) line: 5478
2021-03-27 07:15:10.70 kernel[0]: (IONVMeFamily)
2021-03-27 07:15:10.70 kernel[0]: (IONVMeFamily)
2021-03-27 07:15:10.70 kernel[0]: (IONVMeFamily)
2021-03-27 07:15:10.70 kernel[0]: (IONVMeFamily) virtual IOReturn IONVMeController::CreateSubmissionQueue(uint16_t, uint8_t)::2861:SQ index=0 entrysize=64
2021-03-27 07:15:10.70 kernel[0]: (IONVMeFamily) virtual IOReturn IONVMeController::CreateSubmissionQueue(uint16_t, uint8_t)::2861:SQ index=0 entrysize=64
2021-03-27 07:15:10.70 kernel[0]: (IONVMeFamily) virtual IOReturn IONVMeController::CreateSubmissionQueue(uint16_t, uint8_t)::2861:SQ index=0 entrysize=64
2021-03-27 07:15:10.70 kernel[0]: (IONVMeFamily) virtual IOReturn IONVMeController::CreateSubmissionQueue(uint16_t, uint8_t)::2861:SQ index=0 entrysize=64
2021-03-27 07:15:10.70 kernel[0]: (IONVMeFamily) virtual IOReturn IONVMeController::CreateSubmissionQueue(uint16_t, uint8_t)::2861:SQ index=1 entrysize=64
2021-03-27 07:15:10.70 kernel[0]: (IONVMeFamily) virtual IOReturn IONVMeController::CreateSubmissionQueue(uint16_t, uint8_t)::2861:SQ index=0 entrysize=64
2021-03-27 07:15:10.70 kernel[0]: (IONVMeFamily) virtual IOReturn IONVMeController::CreateSubmissionQueue(uint16_t, uint8_t)::2861:SQ index=1 entrysize=64
2021-03-27 07:15:10.70 kernel[0]: (IONVMeFamily) virtual IOReturn IONVMeController::CreateSubmissionQueue(uint16_t, uint8_t)::2861:SQ index=1 entrysize=64
2021-03-27 07:15:10.71 kernel[0]: (IONVMeFamily) virtual IOReturn IONVMeController::CreateSubmissionQueue(uint16_t, uint8_t)::2861:SQ index=1 entrysize=64
2021-03-27 07:15:10.71 kernel[0]: (IONVMeFamily) virtual IOReturn IONVMeController::CreateSubmissionQueue(uint16_t, uint8_t)::2861:SQ index=0 entrysize=64
2021-03-27 07:15:10.71 kernel[0]: (IONVMeFamily) virtual IOReturn IONVMeController::CreateSubmissionQueue(uint16_t, uint8_t)::2861:SQ index=1 entrysize=64
2021-03-27 07:15:10.71 kernel[0]: (IONVMeFamily) virtual IOReturn IONVMeController::CreateSubmissionQueue(uint16_t, uint8_t)::2861:SQ index=1 entrysize=64
2021-03-27 07:15:10.98 kernel[0]: (IONVMeFamily) line: 5478
2021-03-27 07:15:11.05 kernel[0]: (IONVMeFamily)
2021-03-27 07:15:11.05 kernel[0]: (IONVMeFamily) virtual IOReturn IONVMeController::CreateSubmissionQueue(uint16_t, uint8_t)::2861:SQ index=0 entrysize=64
2021-03-27 07:15:11.06 kernel[0]: (IONVMeFamily) virtual IOReturn IONVMeController::CreateSubmissionQueue(uint16_t, uint8_t)::2861:SQ index=0 entrysize=64
2021-03-27 07:15:11.06 kernel[0]: (IONVMeFamily) virtual IOReturn IONVMeController::CreateSubmissionQueue(uint16_t, uint8_t)::2861:SQ index=1 entrysize=64
2021-03-27 07:15:11.06 kernel[0]: (IONVMeFamily) virtual IOReturn IONVMeController::CreateSubmissionQueue(uint16_t, uint8_t)::2861:SQ index=1 entrysize=64

But it does not panic

From BS 11.2.3:

log show --predicate 'process == "kernel" AND (eventMessage CONTAINS[c] "nvme")' --style syslog --source --last boot
Filtering the log data using "process == "kernel" AND composedMessage CONTAINS[c] "nvme""
Skipping info and debug messages, pass --info and/or --debug to include.
Timestamp                       (process)[PID]    
2021-03-27 09:03:55.144333-0400  localhost kernel[0]: (IONVMeFamily) <IONVMeFamily`IONVMeDebugAssert> AppleNVMe Assert failed: ( 0 != data )
2021-03-27 09:03:55.151607-0400  localhost kernel[0]: (IONVMeFamily) <IONVMeFamily`IONVMeDebugAssert> AppleNVMe Assert failed: ( 0 != data )
2021-03-27 09:03:55.151615-0400  localhost kernel[0]: (IONVMeFamily) <IONVMeFamily`IONVMeDebugAssert> file: /AppleInternal/BuildRoot/Library/Caches/com.apple.xbs/Sources/IONVMeFamily/IONVMeFamily-557.60.1/Common/IONVMeController.cpp
2021-03-27 09:03:55.152760-0400  localhost kernel[0]: (IONVMeFamily) <IONVMeFamily`IONVMeDebugAssert> AppleNVMe Assert failed: ( 0 != data )
2021-03-27 09:03:55.152767-0400  localhost kernel[0]: (IONVMeFamily) <IONVMeFamily`IONVMeDebugAssert> file: /AppleInternal/BuildRoot/Library/Caches/com.apple.xbs/Sources/IONVMeFamily/IONVMeFamily-557.60.1/Common/IONVMeController.cpp
2021-03-27 09:03:55.153907-0400  localhost kernel[0]: (IONVMeFamily) <IONVMeFamily`IONVMeDebugAssert> AppleNVMe Assert failed: 0 == (status)
2021-03-27 09:03:55.153913-0400  localhost kernel[0]: (IONVMeFamily) <IONVMeFamily`IONVMeDebugAssert> file: /AppleInternal/BuildRoot/Library/Caches/com.apple.xbs/Sources/IONVMeFamily/IONVMeFamily-557.60.1/Common/IONVMeController.cpp
2021-03-27 09:03:55.153927-0400  localhost kernel[0]: (IONVMeFamily) <IONVMeFamily`IONVMeDebugAssert> AppleNVMe Assert failed: ( 0 != data )
2021-03-27 09:03:55.153935-0400  localhost kernel[0]: (IONVMeFamily) <IONVMeFamily`IONVMeDebugAssert> file: /AppleInternal/BuildRoot/Library/Caches/com.apple.xbs/Sources/IONVMeFamily/IONVMeFamily-557.60.1/Common/IONVMeController.cpp
2021-03-27 09:03:55.153980-0400  localhost kernel[0]: (IONVMeFamily) <IONVMeFamily`IONVMeDebugAssert> AppleNVMe Assert failed: ( 0 != data )
2021-03-27 09:03:55.153981-0400  localhost kernel[0]: (IONVMeFamily) <IONVMeFamily`IONVMeDebugAssert> AppleNVMe Assert failed: ( 0 != data )
2021-03-27 09:03:55.153987-0400  localhost kernel[0]: (IONVMeFamily) <IONVMeFamily`IONVMeDebugAssert> file: /AppleInternal/BuildRoot/Library/Caches/com.apple.xbs/Sources/IONVMeFamily/IONVMeFamily-557.60.1/Common/IONVMeController.cpp
2021-03-27 09:03:55.153989-0400  localhost kernel[0]: (IONVMeFamily) <IONVMeFamily`IONVMeDebugAssert> file: /AppleInternal/BuildRoot/Library/Caches/com.apple.xbs/Sources/IONVMeFamily/IONVMeFamily-557.60.1/Common/IONVMeController.cpp
2021-03-27 09:03:55.154968-0400  localhost kernel[0]: (IONVMeFamily) <IONVMeFamily`IONVMeDebugAssert> AppleNVMe Assert failed: ( 0 != data )
2021-03-27 09:03:55.154974-0400  localhost kernel[0]: (IONVMeFamily) <IONVMeFamily`IONVMeDebugAssert> file: /AppleInternal/BuildRoot/Library/Caches/com.apple.xbs/Sources/IONVMeFamily/IONVMeFamily-557.60.1/Common/IONVMeController.cpp
2021-03-27 09:03:55.155059-0400  localhost kernel[0]: (IONVMeFamily) <IONVMeFamily`IONVMeDebugAssert> AppleNVMe Assert failed: ( 0 != data )
2021-03-27 09:03:55.155060-0400  localhost kernel[0]: (IONVMeFamily) <IONVMeFamily`IONVMeDebugAssert> AppleNVMe Assert failed: ( 0 != data )
2021-03-27 09:03:55.155066-0400  localhost kernel[0]: (IONVMeFamily) <IONVMeFamily`IONVMeDebugAssert> file: /AppleInternal/BuildRoot/Library/Caches/com.apple.xbs/Sources/IONVMeFamily/IONVMeFamily-557.60.1/Common/IONVMeController.cpp
2021-03-27 09:03:55.155067-0400  localhost kernel[0]: (IONVMeFamily) <IONVMeFamily`IONVMeDebugAssert> file: /AppleInternal/BuildRoot/Library/Caches/com.apple.xbs/Sources/IONVMeFamily/IONVMeFamily-557.60.1/Common/IONVMeController.cpp
2021-03-27 09:03:55.155257-0400  localhost kernel[0]: (IONVMeFamily) <IONVMeFamily`IONVMeController::CreateSubmissionQueue(unsigned short, unsigned char)> virtual IOReturn IONVMeController::CreateSubmissionQueue(uint16_t, uint8_t)::2887:SQ index=0 entrysize=64
2021-03-27 09:03:55.156011-0400  localhost kernel[0]: (IONVMeFamily) <IONVMeFamily`IONVMeDebugAssert> AppleNVMe Assert failed: 0 == (status)
2021-03-27 09:03:55.156018-0400  localhost kernel[0]: (IONVMeFamily) <IONVMeFamily`IONVMeDebugAssert> file: /AppleInternal/BuildRoot/Library/Caches/com.apple.xbs/Sources/IONVMeFamily/IONVMeFamily-557.60.1/Common/IONVMeController.cpp
2021-03-27 09:03:55.156084-0400  localhost kernel[0]: (IONVMeFamily) <IONVMeFamily`IONVMeDebugAssert> AppleNVMe Assert failed: 0 == (status)
2021-03-27 09:03:55.156085-0400  localhost kernel[0]: (IONVMeFamily) <IONVMeFamily`IONVMeDebugAssert> AppleNVMe Assert failed: 0 == (status)
2021-03-27 09:03:55.156093-0400  localhost kernel[0]: (IONVMeFamily) <IONVMeFamily`IONVMeDebugAssert> file: /AppleInternal/BuildRoot/Library/Caches/com.apple.xbs/Sources/IONVMeFamily/IONVMeFamily-557.60.1/Common/IONVMeController.cpp
2021-03-27 09:04:08.848369-0400  localhost kernel[0]: (ntfs) <ntfs`ntfs_system_inodes_get> NTFS volume name WIN10NVME, version 3.1.
2021-03-27 09:05:27.878782-0400  localhost kernel[0]: (Sandbox) <Sandbox`sb_event> Sandbox: CoreSpotlightSer(1002) deny(1) file-read-metadata /Volumes/WIN10NVME
2021-03-27 09:05:27.878963-0400  localhost kernel[0]: (Sandbox) <Sandbox`sb_event> Sandbox: 3 duplicate reports for CoreSpotlightSer deny(1) file-read-metadata /Volumes/WIN10NVME
K-Hobert commented 3 years ago

bootlogs.txt When Mac OS 11.3 beta 5 has a successfully boot on my Mac Pro, noticeable “lag” of 3~ seconds between opencanopy and Mac OS verbose text. Screen turning black for much longer has a become a reliable sign for me.

khronokernel commented 3 years ago

Saw this post, wondering @startergo do you have Thunderbolt3 cards installed by any chance?

https://www.tonymacx86.com/threads/gigabyte-z490-vision-d-thunderbolt-3-i5-10400-amd-rx-580.298642/post-2237496

startergo commented 3 years ago

No I don't have any Thunderbolt cards

dakanji commented 3 years ago

Also the AppleNVMe Assert failed is not the reason for the kernel panic as it is present in Catalina's Bootlog_Kernel: But it does not panic

You would expect problems when ASSERT fails and the Catalina behaviour actually looks like a bug. If so, and it got fixed, then a crash on ASSERT failure will be the expected outcome ... wouldn't it?

Can the ASSERT be patched out?

khronokernel commented 3 years ago

So seeing that 11.3 RC still has these issues, I thought I'd discuss some of our internal findings regarding this situation.

So currently we believe that the issue is not IONVMeFamily related and instead is surrounding IOPCIFamily. Reason we believe this is that non-Mac Pro devices have been experiencing similar boot issues relating to race conditions. Specific culprits:

IONVMeFamily is quite sensitive to hardware responses so assuming IOPCIFamily is being delayed, this would result in the timer kernel panic due to inadequate response from the rest of the OS.

Regarding where to go from here, I'm unsure. This seems to be an accidental bug on Apple's part assuming all hardware would be ready and able to handle all kernel requests, so I do feel this is an issue Apple would need to resolve and less one that we as a community could solve. Assuming IOPCIFamily is the sole culprit, it's impossible to inject an older variant into the OS due to linking issues as well as being at the end of the kernel cache. Instead we would require root volume patches and rebuild the KernelCollection with the rolled back IOPCIFamily

It's a fugly mess in all honesty, and I worry about when Apple may notice these issues. It may be a 11.4 Beta 1 fix or a macOS 12 fix. But as it stands, we can't really do much besides try adding more and more logging to the OS to slow down the boot process as a "work-around"

khronokernel commented 3 years ago

For when macOS 11.3 does release to the public, I've uploaded a copy of 11.2.3 to Archive.org as a work-around to gaining a stable machine:

startergo commented 3 years ago

So currently we believe that the issue is not IONVMeFamily related and instead is surrounding IOPCIFamily.

Is this bug affecting only unsupported hardware? No errors on any of these machines?

These Mac models are compatible with macOS Big Sur:

MacBook (2015 or later)
MacBook Air (2013 or later)
MacBook Pro (Late 2013 or later)
Mac mini (2014 or later)
iMac (2014 or later)
iMac Pro (2017 or later)
Mac Pro (2013 or later)

Some of them even though supported by BS are vintage machines.

khronokernel commented 3 years ago

Is this bug affecting only unsupported hardware?

We believe this should affect users with multiple thunderbolt devices plugged in with Haswell-Broadwell era CPUs however this is currently an assumption. It seems to be heavily single core dependant as Penryn is the most likely to get these boot errors with Arrendale being the next closest. We assume MacPro6,1 should experience these issues as well since Ivy Bridge EP is fairly slow in single core in Apple's overall supported line up.

Without more points of data, hard to determine in all honesty. Only Intel Macs I own are iMac11,2 and MacBook7,1 so my data set is fairly limited besides those commenting in the community as well

startergo commented 3 years ago

Looking at this it looks like some older machines don't have that issue: https://forums.macrumors.com/threads/macos-11-big-sur-on-unsupported-macs-thread.2242172/post-29794242. It must be specific to certain controllers.

khronokernel commented 3 years ago

Potentially, however since multiple types of controllers from different generations, vendors, etc experience these issues, this makes it difficult to understand what would be considered a good PCI device.

If we only experienced NVMe issues, this would make sense as vendors commonly have weird logic and such. However seeing as AHCI, Wifi and Ethernet controllers also experience this issue, I'm a bit puzzled as to what the OS considers a "fast" controller

dakanji commented 3 years ago

@khronokernel ... these older units @startergo pointed out (cMP3,1) do not have NVMe support and use nvme.ffs from the 6,1 injected into their firmware as per this guide: https://docs.google.com/document/d/1WNkM9LuGPq1sArO9EedWBHYq14NU7m-mDBLAWWJipyM/edit#

Could this be a factor in why they work?

khronokernel commented 3 years ago

@dakanji UEFI Driver setup should have little affect during the IOKit probing and kernel extensions starting up. Only way it could affect them is putting the devices into an odd state and therefore requiring extra time for NVMeFamily to slow down and let IOPCIFamily catch up. Only a theory, but tbh I don't have much other explanation besides the NVMe controller itself actually being the limiting factor and the Mac Pro's firmware being a placebo.

khronokernel commented 3 years ago

Curious, has anyone here attempted to downgrade 11.3+'s IOPCIFamily or IONVMeFamily? Currently running some tests with my iMac11,2 on 11.4 B1 and the downgraded IOPCIFamily hasn't resulted in any boot stalls with our ethernet. Still testing however results are promising from our end

I would recommend anyone with capable hardware to install 11.4 on an NVMe drive in a hackintosh or Thunderbolt enclosure with a supported Mac, downgrade IOPCIFamily and then try to boot the machine in the problematic Mac Pro.

For those unfamiliar with Root Volume patching, the following is required:

Once these 2 are set, simply boot macOS and mount the root volume with the following commands (same ones OCLP uses for GPU acceleration patching):

# Find your System volume
diskutil list

# From the below list, we can see our System volume is disk5s5
/dev/disk5 (synthesized):
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:      APFS Container Scheme -                      +255.7 GB   disk5
                                 Physical Store disk4s2
   1:                APFS Volume ⁨Big Sur HD - Data⁩       122.5 GB   disk5s1
   2:                APFS Volume ⁨Preboot⁩                 309.4 MB   disk5s2
   3:                APFS Volume ⁨Recovery⁩                887.8 MB   disk5s3
   4:                APFS Volume ⁨VM⁩                      1.1 MB     disk5s4
   5:                APFS Volume ⁨Big Sur HD⁩              16.2 GB    disk5s5
   6:              APFS Snapshot ⁨com.apple.os.update-...⁩ 16.2 GB    disk5s5s

# Mount the drive(ie. disk5s5)
sudo mount -o nobrowse -t apfs  /dev/disk5s5 /System/Volumes/Update/mnt1

# Remove old kext
sudo rm -R /System/Volumes/Update/mnt1/System/Library/Extensions/IOPCIFamily.kext

# Copy over your kext (change desktop to whether you have the 11.2.3 kext)
sudo cp -R ~/Desktop/IOPCIFamily.kext /System/Volumes/Update/mnt1/System/Library/Extensions/

# Fix permissions
sudo chmod -Rf 755 /System/Volumes/Update/mnt1/System/Library/Extensions/IOPCIFamily.kext
sudo chown -Rf root:wheel /System/Volumes/Update/mnt1/System/Library/Extensions/IOPCIFamily.kext

# Rebuild kernel cache with 11.2.3's IOPCIFamily
sudo kmutil install --volume-root /System/Volumes/Update/mnt1/ --update-all

# Create new bootable snapshot
sudo bless --folder /System/Volumes/Update/mnt1/System/Library/CoreServices --bootefi --create-snapshot
startergo commented 3 years ago

has anyone here attempted to downgrade 11.3+'s IOPCIFamily or IONVMeFamily?

I also downgraded apart from those 2 IOAHCI kexts as well. Did not see a difference. Also I have to mention that this time I installed BS 11.4b1 on an SSD SATA connected to the South bridge. Nevertheless any time an NVMe drive was attached to a PCIE slot I got the same NVMe panic during boot. It looks more likely that it is not the kernel extensions issue or at least not only those one.

BrianAMartin221 commented 3 years ago

For when macOS 11.3 does release to the public, I've uploaded a copy of 11.2.3 to Archive.org as a work-around to gaining a stable machine:

I am an absolute bonehead and upgraded my Mac Pro 5,1 with Opencore today (totally forgot it was a machine I should have waited on), While I have pently of backups id prefer to just get my NVME back up and running, Can you add a linkt as to how to apply that 11.2.3 OS back ton my boot drive?)

BrianAMartin221 commented 3 years ago

For when macOS 11.3 does release to the public, I've uploaded a copy of 11.2.3 to Archive.org as a work-around to gaining a stable machine:

I am an absolute bonehead and upgraded my Mac Pro 5,1 with Opencore today (totally forgot it was a machine I should have waited on), While I have pently of backups id prefer to just get my NVME back up and running, Can you add a linkt as to how to apply that 11.2.3 OS back ton my boot drive?)

Sorry I had a brain fart, i am installing the Image on a bootable USB and will install it on the NVME drive. :)

khronokernel commented 3 years ago

Could anyone confirm if the new NVMe patches in OCLP 0.1.2 fixes the 11.3 boot problems?

lumosideas commented 3 years ago

Could anyone confirm if the new NVMe patches in OCLP 0.1.2 fixes the 11.3 boot problems?

I still have many freeze during the boot on my Mac Pro 3.1 using OCLP 0.1.2. This is my Mac Pro 3.1 configuration:

Mac Pro 3,1 (Early 2008 flashed APFS patched EFI) 2x4cores 2,8Ghz | 32 GB DDR2 PC2-6400F 800MHz ECC FB | Samsung SSD 850 PRO 1TB | Radeon RX 580 8GB | Sonnet Allegro USB-C 3.2 PCIe Card | Broadcom BCM94360CD

a-phe-lei-a commented 3 years ago

For when macOS 11.3 does release to the public, I've uploaded a copy of 11.2.3 to Archive.org as a work-around to gaining a stable machine:

You can download it directly from Apple Servers with softwareupdate --fetch-full-installer --full-installer-version 11.2.3

Ausdauersportler commented 3 years ago

Yes, we know! But usually these Apple download links expire after a while (3-6 months) - so there is not long term guarantee to get from Apple. Since we do not know when and if the problem for the MacPro5,1 users can be solved this is a fall back solution.

bloubum commented 3 years ago

I have a Mac Pro 5,1 with flashed 580 GPU and a Sonnet silent NVME. 11.3 and 11.4 will not boot running on my Sonnet.

lumosideas commented 3 years ago

Hi, I made some test and I notice that OCLP 0.0.22, 0.1.0 are much more stable on boot compared to the latest releases. I notice only one freeze that require a forced power off.

jbinder6342 commented 3 years ago

I'm not fully aware of the layout of things yet but I didn't notice this which might be of interest... https://github.com/acidanthera/NVMeFix

khronokernel commented 3 years ago

As mentioned throughout this thread and on the Macrumors thread, this is not an NVMe issue and is instead a PCI issue. Specifically a race condition on device setup. More devices present in the device tree will result in high chances of boot failures.

A kernel extension would not be able to fix this due to how early of a fault it is, would either need to be resolved by Apple or implement delays in the kernel itself around bsd_init and other init code

ylluminate commented 3 years ago

A kernel extension would not be able to fix this due to how early of a fault it is, ... or implement delays in the kernel itself around bsd_init and other init code

Is this possible through some kind of OpenCore injection(s) into the kernel at this point?

khronokernel commented 3 years ago

As this issue and the MacRumors thread have been derailed, I've opened a new issue with consolidated information:

https://github.com/dortania/OpenCore-Legacy-Patcher/issues/358

Issue is locked as it's meant for proper development, use the MacRumors forum for more casual discussion

ylluminate commented 3 years ago

Thank you for the concerted and focused issue, information and hope of moving through this.