Thinkpad Bug: Kernel panic at boot in "kec.corecrypto" when trying to run the Big Sur image on a Thinkpad T460s

[3xr]

Hello,

When trying to run the Big Sur image using the following command, after having installed at the requirements and dependencies:

docker run -it \
    --device /dev/kvm \
    -p 50922:10022 \
    -e DEVICE_MODEL="iMacPro1,1" \
    -e WIDTH=1300 \
    -e HEIGHT=700 \
    -v /tmp/.X11-unix:/tmp/.X11-unix \
    -e "DISPLAY=${DISPLAY:-:0.0}" \
    -e GENERATE_UNIQUE=true \
    -e EXTRA="-device virtio-serial-pci -device usb-host,hostbus=1,hostport=4" \
    -e MASTER_PLIST_URL=https://raw.githubusercontent.com/sickcodes/osx-serial-generator/master/config-custom.plist \
    sickcodes/docker-osx:big-sur

(The USB forwarding directive is just for forwarding a MIDI keyboard, I can try without it if needed (Edit: same thing without it))

The QEMU window spawns, and I can the select the macOS disk as a boot device, but after 1-2 minutes I'm getting a kernel panic about something happening in the com.apple.kec.corecrypto(11.1) module of macOS:

And before that I have this screen (outlining an "Err(0xE) <- EB.WL.PWLFNV - RT.GV failure error)?

I'm using a standard Lenovo Thinkpad T460s (2016) on Ubuntu 20.04 LTS with extended RAM and SSD, and with the original Intel CPU and iGPU.

Would you have any idea about the possible cause of the issue?

Regards,

Debug logs

$ uname -a \
> ; echo "${DISPLAY}" \
> ; echo 1 | sudo tee /sys/module/kvm/parameters/ignore_msrs \
> ; grep NAME /etc/os-release \
> ; df -h . \
> ; qemu-system-x86_64 --version \
> ; libvirtd --version \
> ; free -mh \
> ; nproc \
> ; egrep -c '(svm|vmx)' /proc/cpuinfo \
> ; ls -lha /dev/kvm \
> ; ls -lha /tmp/.X11-unix/ \
> ; ps aux | grep dockerd \
> ; docker ps | grep osx \
> ; grep "docker\|kvm\|virt" /etc/group
Linux myusername-ThinkPad-T460s 5.8.0-59-generic #66~20.04.1-Ubuntu SMP Thu Jun 17 11:14:10 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
:0
[sudo] Mot de passe de myusername : 
1
NAME="Ubuntu"
PRETTY_NAME="Ubuntu 20.04.2 LTS"
VERSION_CODENAME=focal
UBUNTU_CODENAME=focal
Sys. de fichiers          Taille Utilisé Dispo Uti% Monté sur
/dev/mapper/vgubuntu-root   914G    352G  516G  41% /
QEMU emulator version 4.2.1 (Debian 1:4.2-3ubuntu6.16)
Copyright (c) 2003-2019 Fabrice Bellard and the QEMU Project developers
libvirtd (libvirt) 6.0.0
              total       utilisé      libre     partagé tamp/cache   disponible
Mem:           19Gi        11Gi       294Mi       2,3Gi       7,8Gi       5,5Gi
Partition d'échange:       975Mi       956Mi        19Mi
4
8
crw-rw----+ 1 root kvm 10, 232 juin  29 23:59 /dev/kvm
total 52K
drwxrwxrwt  2 root  root  4,0K juin  29 23:59 .
drwxrwxrwt 34 root  root   44K juil.  3 10:09 ..
srwxrwxrwx  1 myusername myusername    0 juin  29 23:59 X0
root      115319  0.3  0.3 1452676 61836 ?       Ssl  juil.02   6:19 /usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock
myusername     409993  0.0  0.0  11608  2644 pts/65   S+   10:19   0:00 grep --color=auto dockerd
560c1f0800fd        sickcodes/docker-osx:big-sur   "/bin/bash -c 'sudo …"   18 minutes ago      Up 18 minutes       0.0.0.0:50922->10022/tcp                           nostalgic_hypatia
kvm:x:108:myusername
docker:x:142:myusername
libvirt:x:146:myusername
libvirt-qemu:x:64055:libvirt-qemu
libvirt-dnsmasq:x:147:

[3xr]

I encounter the same issue with the Catalina image, while we can see a larger backtrace thanks to the higher screen resolution:

I deleted the former Docker images and ran the following command this time:

date ; time docker run -i \
    --device /dev/kvm \
    -p 50922:10022 \
    -e DEVICE_MODEL="iMacPro1,1" \
    -e WIDTH=1300 \
    -e HEIGHT=700 \
    -v /tmp/.X11-unix:/tmp/.X11-unix \
    -e "DISPLAY=${DISPLAY:-:0.0}" \
    -e EXTRA="-device virtio-serial-pci -device usb-host,hostbus=1,hostport=4" \
    sickcodes/docker-osx:latest

[sickcodes]

Try this style of USB passthrough:

-e EXTRA='-device usb-host,hostbus=1,hostaddr=4' \

[3xr]

Hello,

As I precised above, removing the USB forwarding line changes nothing at all.

As you can see in the backtrace, this "Unknown Opcode" CPU fault seems to lay in the _gcm_init routine of Apple's CoreCrypto library, which seems to depend both on the SSSE3 and AES-NI instruction extension sets.

My host CPU (Intel Core i7-6600U) correctly supports both (it doesn't support AVX-512 which doesn't seem to be known to be involved here).

$ grep ssse3 /proc/cpuinfo | head -1
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb invpcid_single pti ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm mpx rdseed adx smap clflushopt intel_pt xsaveopt xsavec xgetbv1 xsaves dtherm ida arat pln pts hwp hwp_notify hwp_act_window hwp_epp md_clear flush_l1d

I think that it would be good to disassemble the Catalina kernel at the address provided by the backtrace in order to perform helpful investigation, but I don't know how to extract it. Maybe that the CR registers give a hint with this kind of exception?

An important thing to note is that while both the Big Sur and Catalina images provided by Docker-OSX fail with this error at boot, sosumi which also relies on QEMU (but with a different stack behind) boots Catalina well on my machine.

Your CPU configuration lines doesn't seem to contain something extravagant to me, although I'm not a QEMU expert. Maybe that these should be compared with more attention or detail with other macOS virtualization scripts.

Regards,

[3xr]

Actually the code simply lays in its own kernel extension file (the macOS Base System/System/Library/Extensions/corecrypto.kext/Contents/MacOS/corecrypto file from the diff/home/arch/OSX-KVM/BaseSystem.img image from the container).

Actually the code at _gcm_init + 8e is not SSSE3 or AES-NI (SSSE3 instructions like pshufb or SSE2 instructions like movdqa or pslldq perform well according to the position in the control flow), it's a pclmulqdq from the CLMUL (2010) instruction set extension.

Brace yourself for IDA screenshot

An instruction set extension that my processor indicates to support, obviously

$ grep pclmulqdq /proc/cpuinfo | head -1 flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb invpcid_single pti ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm mpx rdseed adx smap clflushopt intel_pt xsaveopt xsavec xgetbv1 xsaves dtherm ida arat pln pts hwp hwp_notify hwp_act_window hwp_epp md_clear flush_l1d

I wonder why the crash, then.

[3xr]

Additional insight:

• Here is the QEMU command run by sosumi (non-panicking)

/snap/sosumi/18/usr/bin/qemu-system-x86_64 -enable-kvm -m 2G -machine q35,accel=kvm -smp 4,cores=2 -cpu Penryn,vendor=GenuineIntel,kvm=on,+sse3,+sse4.2,+aes,+xsave,+avx,+xsaveopt,+xsavec,+xgetbv1,+avx2,+bmi2,+smep,+bmi1,+fma,+movbe,+invtsc -device isa-applesmc,osk=ourhardworkbythesewordsguardedpleasedontsteal(c)AppleComputerInc -smbios type=2 -object rng-random,id=rng0,filename=/dev/urandom -device virtio-rng-pci,rng=rng0 -serial mon:stdio -drive if=pflash,format=raw,readonly,file=/snap/sosumi/18/firmware/OVMF_CODE.fd -drive if=pflash,format=raw,file=/home/myusername/snap/sosumi/common/firmware/OVMF_VARS-1024x768.fd -device virtio-vga,virgl=on -display sdl,gl=on -L /snap/sosumi/18/usr/share/seabios/ -L /snap/sosumi/18/usr/lib/ipxe/qemu/ -audiodev pa,id=pa,server=/run/user/1000/pulse/native -device ich9-intel-hda -device hda-output,audiodev=pa -usb -device usb-kbd -device usb-mouse -netdev user,id=net0 -device vmxnet3,netdev=net0,id=net0 -drive id=ESP,if=virtio,format=qcow2,file=/home/myusername/snap/sosumi/common/ESP.qcow2 -drive id=SystemDisk,if=virtio,file=/home/myusername/snap/sosumi/common/macos.qcow2 -device usb-host,vendorid=0x0763,productid=0x3111,guest-reset=false

• Here is the QEMU command run by Docker-OSX (panicking at boot, as explained above)

exec qemu-system-x86_64 -m 3000 -cpu Penryn,vendor=GenuineIntel,+invtsc,vmware-cpuid-freq=on,+pcid,+ssse3,+sse4.2,+popcnt,+avx,+aes,+xsave,+xsaveopt,check, -machine q35,accel=kvm:tcg -smp 4,cores=4 -usb -device usb-kbd -device usb-tablet -device 'isa-applesmc,osk=ourhardworkbythesewordsguardedpleasedontsteal(c)AppleComputerInc' -drive if=pflash,format=raw,readonly,file=/home/arch/OSX-KVM/OVMF_CODE.fd -drive if=pflash,format=raw,file=/home/arch/OSX-KVM/OVMF_VARS-1024x768.fd -smbios type=2 -audiodev alsa,id=hda -device ich9-intel-hda -device hda-duplex,audiodev=hda -device ich9-ahci,id=sata -drive id=OpenCoreBoot,if=none,snapshot=on,format=qcow2,file=/home/arch/OSX-KVM/OpenCore-Catalina/OpenCore.qcow2 -device ide-hd,bus=sata.2,drive=OpenCoreBoot -device ide-hd,bus=sata.3,drive=InstallMedia -drive id=InstallMedia,if=none,file=/home/arch/OSX-KVM/BaseSystem.img,format=qcow2 -drive id=MacHDD,if=none,file=/home/arch/OSX-KVM/mac_hdd_ng.img,format=qcow2 -device ide-hd,bus=sata.4,drive=MacHDD -netdev user,id=net0,hostfwd=tcp::10022-:22,hostfwd=tcp::5900-:5900, -device vmxnet3,netdev=net0,id=net0,mac=52:54:00:09:49:17 -monitor stdio -vga vmware

The corecrypto kernel extension binary from sosumi's image also has a pclmulqdq instruction at _gcm_init + 8e.

[3xr]

It looks like it may be linked to the +ssse3 flag in the Docker-OSX configuration, which is not present in Sosumi`'s. The CoreCrypto module does a CPUID flag check on this feature just before calling. ^^

See: https://opensource.apple.com/source/xnu/xnu-1504.9.17/osfmk/i386/cpuid.h.auto.html

#define CPUID_FEATURE_SSSE3   _HBit(9)  /* Supplemental SSE3 instructions */

I'd suggest to align on the -cpu flags of Sosumi and remove the +ssse3 (see the command above). (Or maybe that adding the +pclmulqdq flag as specified in qemu-system-x86_64 -cpu help would change something, if it does not only affect the returned cpuid.)

[sickcodes]

This is excellent, thank you for posting all these, I'll remove the offending flag in about 20 minutes 😁

[sickcodes]

RE: Sosumi boot args:

Also there are some changes made recently upstream in KVM-Opencore to prepare for Monterrey that I haven't merged yet as I haven't had time to test:

https://raw.githubusercontent.com/sickcodes/osx-serial-generator/v13/config-custom.plist

As seen in:

https://github.com/thenickdude/KVM-Opencore/releases/tag/v13

Changes ready to merge: https://github.com/sickcodes/osx-serial-generator/pull/13

Also as we speak, I'm working on https://github.com/sickcodes/dock-droid and lots of random qemu args that we currently have can be removed, namely due to QEMU 6, but also older stuff.

For example, simply using:

    -cpu host \
    -smp 8 \

And also adding:

    -display sdl,gl=on \

Testing right now

https://github.com/sickcodes/osx-serial-generator/pull/13

[3xr]

Hello,

Regarding your last diff, is this a change that you done or a change that you are projecting to do? It seems that the mentions of CPU configuration in your repository didn't change yet.

Regards

[sickcodes]

Added (just now) in https://github.com/sickcodes/Docker-OSX/commit/866a429848e8bca422a10712dae0711cbbdad99b

You can now use:

-e CPUID_FLAGS='vendor=GenuineIntel,+invtsc,vmware-cpuid-freq=on,+pcid,+ssse3,+sse4.2,+popcnt,+avx,+aes,+xsave,+xsaveopt,check,' \

[idzikovski]

I am having this issue as well. Is there any way to overcome this?

[sickcodes]

Can you let me know the name of the CPU you are using?

Please post the output of: qemu-system-x86_64 -cpu help

For example, here are mine:

Recognized CPUID flags:
  3dnow 3dnowext 3dnowprefetch abm ace2 ace2-en acpi adx aes amd-no-ssb
  amd-ssbd amd-stibp apic arat arch-capabilities avic avx avx2
  avx512-4fmaps avx512-4vnniw avx512-bf16 avx512-fp16 avx512-vp2intersect
  avx512-vpopcntdq avx512bitalg avx512bw avx512cd avx512dq avx512er avx512f
  avx512ifma avx512pf avx512vbmi avx512vbmi2 avx512vl avx512vnni bmi1 bmi2
  bus-lock-detect cid cldemote clflush clflushopt clwb clzero cmov
  cmp-legacy core-capability cr8legacy cx16 cx8 dca de decodeassists ds
  ds-cpl dtes64 erms est extapic f16c flushbyasid fma fma4 fpu fsgsbase
  fsrm full-width-write fxsr fxsr-opt gfni hle ht hypervisor ia64 ibpb ibrs
  ibrs-all ibs intel-pt intel-pt-lip invpcid invtsc kvm-asyncpf
  kvm-asyncpf-int kvm-hint-dedicated kvm-mmu kvm-msi-ext-dest-id
  kvm-nopiodelay kvm-poll-control kvm-pv-eoi kvm-pv-ipi kvm-pv-sched-yield
  kvm-pv-tlb-flush kvm-pv-unhalt kvm-steal-time kvmclock kvmclock
  kvmclock-stable-bit la57 lahf-lm lbrv lm lwp mca mce md-clear mds-no
  misalignsse mmx mmxext monitor movbe movdir64b movdiri mpx msr mtrr
  nodeid-msr npt nrip-save nx osvw pae pat pause-filter pbe pcid pclmulqdq
  pcommit pdcm pdpe1gb perfctr-core perfctr-nb pfthreshold pge phe phe-en
  pks pku pmm pmm-en pn pni popcnt pschange-mc-no pse pse36 rdctl-no rdpid
  rdrand rdseed rdtscp rsba rtm sep serialize sha-ni skinit
  skip-l1dfl-vmentry smap smep smx spec-ctrl split-lock-detect ss ssb-no
  ssbd sse sse2 sse4.1 sse4.2 sse4a ssse3 stibp svm svm-lock svme-addr-chk
  syscall taa-no tbm tce tm tm2 topoext tsc tsc-adjust tsc-deadline
  tsc-scale tsx-ctrl tsx-ldtrk umip v-vmsave-vmload vaes vgif virt-ssbd
  vmcb-clean vme vmx vmx-activity-hlt vmx-activity-shutdown
  vmx-activity-wait-sipi vmx-apicv-register vmx-apicv-vid vmx-apicv-x2apic
  vmx-apicv-xapic vmx-cr3-load-noexit vmx-cr3-store-noexit
  vmx-cr8-load-exit vmx-cr8-store-exit vmx-desc-exit vmx-encls-exit
  vmx-entry-ia32e-mode vmx-entry-load-bndcfgs vmx-entry-load-efer
  vmx-entry-load-pat vmx-entry-load-perf-global-ctrl vmx-entry-load-pkrs
  vmx-entry-load-rtit-ctl vmx-entry-noload-debugctl vmx-ept vmx-ept-1gb
  vmx-ept-2mb vmx-ept-advanced-exitinfo vmx-ept-execonly vmx-eptad
  vmx-eptp-switching vmx-exit-ack-intr vmx-exit-clear-bndcfgs
  vmx-exit-clear-rtit-ctl vmx-exit-load-efer vmx-exit-load-pat
  vmx-exit-load-perf-global-ctrl vmx-exit-load-pkrs
  vmx-exit-nosave-debugctl vmx-exit-save-efer vmx-exit-save-pat
  vmx-exit-save-preemption-timer vmx-flexpriority vmx-hlt-exit vmx-ins-outs
  vmx-intr-exit vmx-invept vmx-invept-all-context vmx-invept-single-context
  vmx-invept-single-context vmx-invept-single-context-noglobals
  vmx-invlpg-exit vmx-invpcid-exit vmx-invvpid vmx-invvpid-all-context
  vmx-invvpid-single-addr vmx-io-bitmap vmx-io-exit vmx-monitor-exit
  vmx-movdr-exit vmx-msr-bitmap vmx-mtf vmx-mwait-exit vmx-nmi-exit
  vmx-page-walk-4 vmx-page-walk-5 vmx-pause-exit vmx-ple vmx-pml
  vmx-posted-intr vmx-preemption-timer vmx-rdpmc-exit vmx-rdrand-exit
  vmx-rdseed-exit vmx-rdtsc-exit vmx-rdtscp-exit vmx-secondary-ctls
  vmx-shadow-vmcs vmx-store-lma vmx-true-ctls vmx-tsc-offset
  vmx-unrestricted-guest vmx-vintr-pending vmx-vmfunc
  vmx-vmwrite-vmexit-fields vmx-vnmi vmx-vnmi-pending vmx-vpid
  vmx-wbinvd-exit vmx-xsaves vmx-zero-len-inject vpclmulqdq waitpkg
  wbnoinvd wdt x2apic xcrypt xcrypt-en xgetbv1 xop xsave xsavec xsaveerptr
  xsaveopt xsaves xstore xstore-en xtpr

[idzikovski]

`Available CPUs: x86 486 (alias configured by machine type)
x86 486-v1
x86 Broadwell (alias configured by machine type)
x86 Broadwell-IBRS (alias of Broadwell-v3)
x86 Broadwell-noTSX (alias of Broadwell-v2)
x86 Broadwell-noTSX-IBRS (alias of Broadwell-v4)
x86 Broadwell-v1 Intel Core Processor (Broadwell)
x86 Broadwell-v2 Intel Core Processor (Broadwell, no TSX)
x86 Broadwell-v3 Intel Core Processor (Broadwell, IBRS)
x86 Broadwell-v4 Intel Core Processor (Broadwell, no TSX, IBRS)
x86 Cascadelake-Server (alias configured by machine type)
x86 Cascadelake-Server-noTSX (alias of Cascadelake-Server-v3)
x86 Cascadelake-Server-v1 Intel Xeon Processor (Cascadelake)
x86 Cascadelake-Server-v2 Intel Xeon Processor (Cascadelake)
x86 Cascadelake-Server-v3 Intel Xeon Processor (Cascadelake)
x86 Conroe (alias configured by machine type)
x86 Conroe-v1 Intel Celeron_4x0 (Conroe/Merom Class Core 2)
x86 Cooperlake (alias configured by machine type)
x86 Cooperlake-v1 Intel Xeon Processor (Cooperlake)
x86 Denverton (alias configured by machine type)
x86 Denverton-v1 Intel Atom Processor (Denverton)
x86 Dhyana (alias configured by machine type)
x86 Dhyana-v1 Hygon Dhyana Processor
x86 EPYC (alias configured by machine type)
x86 EPYC-IBPB (alias of EPYC-v2)
x86 EPYC-Milan (alias configured by machine type)
x86 EPYC-Milan-v1 AMD EPYC-Milan Processor
x86 EPYC-Rome (alias configured by machine type)
x86 EPYC-Rome-v1 AMD EPYC-Rome Processor
x86 EPYC-Rome-v2 AMD EPYC-Rome Processor
x86 EPYC-v1 AMD EPYC Processor
x86 EPYC-v2 AMD EPYC Processor (with IBPB)
x86 EPYC-v3 AMD EPYC Processor
x86 Haswell (alias configured by machine type)
x86 Haswell-IBRS (alias of Haswell-v3)
x86 Haswell-noTSX (alias of Haswell-v2)
x86 Haswell-noTSX-IBRS (alias of Haswell-v4)
x86 Haswell-v1 Intel Core Processor (Haswell)
x86 Haswell-v2 Intel Core Processor (Haswell, no TSX)
x86 Haswell-v3 Intel Core Processor (Haswell, IBRS)
x86 Haswell-v4 Intel Core Processor (Haswell, no TSX, IBRS)
x86 Icelake-Client (alias configured by machine type)
x86 Icelake-Client-noTSX (alias of Icelake-Client-v2)
x86 Icelake-Client-v1 Intel Core Processor (Icelake)
x86 Icelake-Client-v2 Intel Core Processor (Icelake)
x86 Icelake-Server (alias configured by machine type)
x86 Icelake-Server-noTSX (alias of Icelake-Server-v2)
x86 Icelake-Server-v1 Intel Xeon Processor (Icelake)
x86 Icelake-Server-v2 Intel Xeon Processor (Icelake)
x86 IvyBridge (alias configured by machine type)
x86 IvyBridge-IBRS (alias of IvyBridge-v2)
x86 IvyBridge-v1 Intel Xeon E3-12xx v2 (Ivy Bridge)
x86 IvyBridge-v2 Intel Xeon E3-12xx v2 (Ivy Bridge, IBRS)
x86 KnightsMill (alias configured by machine type)
x86 KnightsMill-v1 Intel Xeon Phi Processor (Knights Mill)
x86 Nehalem (alias configured by machine type)
x86 Nehalem-IBRS (alias of Nehalem-v2)
x86 Nehalem-v1 Intel Core i7 9xx (Nehalem Class Core i7)
x86 Nehalem-v2 Intel Core i7 9xx (Nehalem Core i7, IBRS update) x86 Opteron_G1 (alias configured by machine type)
x86 Opteron_G1-v1 AMD Opteron 240 (Gen 1 Class Opteron)
x86 Opteron_G2 (alias configured by machine type)
x86 Opteron_G2-v1 AMD Opteron 22xx (Gen 2 Class Opteron)
x86 Opteron_G3 (alias configured by machine type)
x86 Opteron_G3-v1 AMD Opteron 23xx (Gen 3 Class Opteron)
x86 Opteron_G4 (alias configured by machine type)
x86 Opteron_G4-v1 AMD Opteron 62xx class CPU
x86 Opteron_G5 (alias configured by machine type)
x86 Opteron_G5-v1 AMD Opteron 63xx class CPU
x86 Penryn (alias configured by machine type)
x86 Penryn-v1 Intel Core 2 Duo P9xxx (Penryn Class Core 2)
x86 SandyBridge (alias configured by machine type)
x86 SandyBridge-IBRS (alias of SandyBridge-v2)
x86 SandyBridge-v1 Intel Xeon E312xx (Sandy Bridge)
x86 SandyBridge-v2 Intel Xeon E312xx (Sandy Bridge, IBRS update)
x86 Skylake-Client (alias configured by machine type)
x86 Skylake-Client-IBRS (alias of Skylake-Client-v2)
x86 Skylake-Client-noTSX-IBRS (alias of Skylake-Client-v3)
x86 Skylake-Client-v1 Intel Core Processor (Skylake)
x86 Skylake-Client-v2 Intel Core Processor (Skylake, IBRS)
x86 Skylake-Client-v3 Intel Core Processor (Skylake, IBRS)
x86 Skylake-Server (alias configured by machine type)
x86 Skylake-Server-IBRS (alias of Skylake-Server-v2)
x86 Skylake-Server-noTSX-IBRS (alias of Skylake-Server-v3)
x86 Skylake-Server-v1 Intel Xeon Processor (Skylake)
x86 Skylake-Server-v2 Intel Xeon Processor (Skylake, IBRS)
x86 Skylake-Server-v3 Intel Xeon Processor (Skylake, IBRS)
x86 Snowridge (alias configured by machine type)
x86 Snowridge-v1 Intel Atom Processor (SnowRidge)
x86 Snowridge-v2 Intel Atom Processor (Snowridge, no MPX)
x86 Westmere (alias configured by machine type)
x86 Westmere-IBRS (alias of Westmere-v2)
x86 Westmere-v1 Westmere E56xx/L56xx/X56xx (Nehalem-C)
x86 Westmere-v2 Westmere E56xx/L56xx/X56xx (IBRS update)
x86 athlon (alias configured by machine type)
x86 athlon-v1 QEMU Virtual CPU version 2.5+
x86 core2duo (alias configured by machine type)
x86 core2duo-v1 Intel(R) Core(TM)2 Duo CPU T7700 @ 2.40GHz x86 coreduo (alias configured by machine type)
x86 coreduo-v1 Genuine Intel(R) CPU T2600 @ 2.16GHz x86 kvm32 (alias configured by machine type)
x86 kvm32-v1 Common 32-bit KVM processor
x86 kvm64 (alias configured by machine type)
x86 kvm64-v1 Common KVM processor
x86 n270 (alias configured by machine type)
x86 n270-v1 Intel(R) Atom(TM) CPU N270 @ 1.60GHz
x86 pentium (alias configured by machine type)
x86 pentium-v1
x86 pentium2 (alias configured by machine type)
x86 pentium2-v1
x86 pentium3 (alias configured by machine type)
x86 pentium3-v1
x86 phenom (alias configured by machine type)
x86 phenom-v1 AMD Phenom(tm) 9550 Quad-Core Processor
x86 qemu32 (alias configured by machine type)
x86 qemu32-v1 QEMU Virtual CPU version 2.5+
x86 qemu64 (alias configured by machine type)
x86 qemu64-v1 QEMU Virtual CPU version 2.5+
x86 base base CPU model type with no features enabled
x86 host KVM processor with all supported host features
x86 max Enables all features supported by the accelerator in the current host

Recognized CPUID flags: 3dnow 3dnowext 3dnowprefetch abm ace2 ace2-en acpi adx aes amd-no-ssb amd-ssbd amd-stibp apic arat arch-capabilities avic avx avx2 avx512-4fmaps avx512-4vnniw avx512-bf16 avx512-vpopcntdq avx512bitalg avx512bw avx512cd avx512dq avx512er avx512f avx512ifma avx512pf avx512vbmi avx512vbmi2 avx512vl avx512vnni bmi1 bmi2 cid cldemote clflush clflushopt clwb clzero cmov cmp-legacy core-capability cr8legacy cx16 cx8 dca de decodeassists ds ds-cpl dtes64 erms est extapic f16c flushbyasid fma fma4 fpu fsgsbase fsrm fxsr fxsr-opt gfni hle ht hypervisor ia64 ibpb ibrs ibrs-all ibs intel-pt invpcid invtsc kvm-asyncpf kvm-hint-dedicated kvm-mmu kvm-nopiodelay kvm-poll-control kvm-pv-eoi kvm-pv-ipi kvm-pv-sched-yield kvm-pv-tlb-flush kvm-pv-unhalt kvm-steal-time kvmclock kvmclock kvmclock-stable-bit la57 lahf-lm lbrv lm lwp mca mce md-clear mds-no misalignsse mmx mmxext monitor movbe movdir64b movdiri mpx msr mtrr nodeid-msr npt nrip-save nx osvw pae pat pause-filter pbe pcid pclmulqdq pcommit pdcm pdpe1gb perfctr-core perfctr-nb pfthreshold pge phe phe-en pku pmm pmm-en pn pni popcnt pschange-mc-no pse pse36 rdctl-no rdpid rdrand rdseed rdtscp rsba rtm sep sha-ni skinit skip-l1dfl-vmentry smap smep smx spec-ctrl split-lock-detect ss ssb-no ssbd sse sse2 sse4.1 sse4.2 sse4a ssse3 stibp svm svm-lock svme-addr-chk syscall taa-no tbm tce tm tm2 topoext tsc tsc-adjust tsc-deadline tsc-scale tsx-ctrl umip v-vmsave-vmload vaes vgif virt-ssbd vmcb-clean vme vmx vmx-activity-hlt vmx-activity-shutdown vmx-activity-wait-sipi vmx-apicv-register vmx-apicv-vid vmx-apicv-x2apic vmx-apicv-xapic vmx-cr3-load-noexit vmx-cr3-store-noexit vmx-cr8-load-exit vmx-cr8-store-exit vmx-desc-exit vmx-encls-exit vmx-entry-ia32e-mode vmx-entry-load-bndcfgs vmx-entry-load-efer vmx-entry-load-pat vmx-entry-load-perf-global-ctrl vmx-entry-load-rtit-ctl vmx-entry-noload-debugctl vmx-ept vmx-ept-1gb vmx-ept-2mb vmx-ept-advanced-exitinfo vmx-ept-execonly vmx-eptad vmx-eptp-switching vmx-exit-ack-intr vmx-exit-clear-bndcfgs vmx-exit-clear-rtit-ctl vmx-exit-load-efer vmx-exit-load-pat vmx-exit-load-perf-global-ctrl vmx-exit-nosave-debugctl vmx-exit-save-efer vmx-exit-save-pat vmx-exit-save-preemption-timer vmx-flexpriority vmx-hlt-exit vmx-ins-outs vmx-intr-exit vmx-invept vmx-invept-all-context vmx-invept-single-context vmx-invept-single-context vmx-invept-single-context-noglobals vmx-invlpg-exit vmx-invpcid-exit vmx-invvpid vmx-invvpid-all-context vmx-invvpid-single-addr vmx-io-bitmap vmx-io-exit vmx-monitor-exit vmx-movdr-exit vmx-msr-bitmap vmx-mtf vmx-mwait-exit vmx-nmi-exit vmx-page-walk-4 vmx-page-walk-5 vmx-pause-exit vmx-ple vmx-pml vmx-posted-intr vmx-preemption-timer vmx-rdpmc-exit vmx-rdrand-exit vmx-rdseed-exit vmx-rdtsc-exit vmx-rdtscp-exit vmx-secondary-ctls vmx-shadow-vmcs vmx-store-lma vmx-true-ctls vmx-tsc-offset vmx-unrestricted-guest vmx-vintr-pending vmx-vmfunc vmx-vmwrite-vmexit-fields vmx-vnmi vmx-vnmi-pending vmx-vpid vmx-wbinvd-exit vmx-xsaves vmx-zero-len-inject vpclmulqdq waitpkg wbnoinvd wdt x2apic xcrypt xcrypt-en xgetbv1 xop xsave xsavec xsaveerptr xsaveopt xsaves xstore xstore-en xtpr `

[sickcodes]

I was able to reproduce this issue when the host does not have KVM.

docker run -it \
    -e CPU=Penryn \
    -e CPUID_FLAGS= \
    -e KVM= \
    -p 50922:10022 \
    -v /tmp/.X11-unix:/tmp/.X11-unix \
    -e "DISPLAY=${DISPLAY:-:0.0}" \
    sickcodes/docker-osx:latest

This produces the halt reboot error, so yes this bug is related to CPUID_FLAGS, any ideas which one is the culprit and if it's required for all systems? I was going to do trial and error till I found it using

-e CPUID_FLAGS= \

[sickcodes]

For contrast and to emphasize the CPUID flag importance, this passes the start but seg faults right after.

docker run -it \
    -e CPU=Penryn \
    -e CPUID_FLAGS="vendor=GenuineIntel," \
    -e KVM= \
    -p 50922:10022 \
    -v /tmp/.X11-unix:/tmp/.X11-unix \
    -e "DISPLAY=${DISPLAY:-:0.0}" \
    sickcodes/docker-osx:latest

[idzikovski]

any ideas which one is the culprit and if it's required for all systems?

Nope, I am not that knowledgable. I am simply trying to run this image using the instructions in the readme. My goal is to do a proof of concept for a mac build agent for CI/CD running in docker

[sickcodes]

Using -e CPU=host \ instead of Penryn may work.

I am able to replicate the exact error that @3xr displayed above and it's only occurring when qemu is unallowed to use /dev/kvm or when it's not passed in on my host or when I remove all the CPUID flags by setting the variable to nothing.

Can you confirm it's doing it with the pure vanilla run command?

docker run -it \
    --device /dev/kvm \
    -p 50922:10022 \
    -v /tmp/.X11-unix:/tmp/.X11-unix \
    -e "DISPLAY=${DISPLAY:-:0.0}" \
    sickcodes/docker-osx:latest

[idzikovski]

With this command I didn't get the panic but I encountered the issue where the mouse gets stuck in the screen where you can choose between disk utility and reinstall

[sickcodes]

Mouse is unusable during the first screen which is normal, buy you can use the arrow keys

[3xr]

it's only occurring when qemu is unallowed to use /dev/kvm or when it's not passed in on my host

Thanks for the remarks!

Actually it seems that it may be linked that the --userns-remap=default option is enabled on my Docker setup, and that the dockremap user which is created automatically by Docker in such conditions was not added to the kvm or libvirt groups.

$ cat /etc/docker/daemon.json 
{
    "userns-remap": "default"
}
$ grep dockremap /etc/passwd
dockremap:x:131:143::/home/dockremap:/bin/false

I have tried to add the dockremap user to the kvm group, but I can't download a truckload of container images to re-test it right now as I'm on a countryside connection.

[sickcodes]

Note with Thinkpads the following from https://dortania.github.io/docs/latest/Configuration.html

ForceOcWriteFlash Type: plist boolean Failsafe: false Description: Enables writing to flash memory for all OpenCore system variables.

Note: This value should be disabled on most types of firmware but is left configurable to account for firmware that may have issues with volatile variable storage overflows or similar. Boot issues across multiple OSes can be observed on e.g. Lenovo Thinkpad T430 and T530 without this quirk. Apple variables related to Secure Boot and hibernation are exempt from this for security reasons. Furthermore, some OpenCore variables are exempt for different reasons, such as the boot log due to an available user option, and the TSC frequency due to timing issues. When toggling this option, a NVRAM reset may be required to ensure full functionality.

[sickcodes]

Possibly related https://github.com/reenigneorcim/SurPlus

[murilopereirame]

Using -e CPU=host \ instead of Penryn may work.

I am able to replicate the exact error that @3xr displayed above and it's only occurring when qemu is unallowed to use /dev/kvm or when it's not passed in on my host or when I remove all the CPUID flags by setting the variable to nothing.

Can you confirm it's doing it with the pure vanilla run command?

docker run -it \
    --device /dev/kvm \
    -p 50922:10022 \
    -v /tmp/.X11-unix:/tmp/.X11-unix \
    -e "DISPLAY=${DISPLAY:-:0.0}" \
    sickcodes/docker-osx:latest

Using the flag -e CPU=host solves the problem of corecrypto but maybe throw a permission error related to KVM, not big deal, just adjust the permissions of /dev/kvm and re-run the command

[JAicewizard]

See https://github.com/sickcodes/Docker-OSX/issues/380#issuecomment-2143425540, maybe also resolves this issue. Also running a thinkpad bug a t14 amd gen 1, for me a bios update did it.