Wouter1 / EMU-driver

OSX Kernel extension for Creative Labs EMU driver
189 stars 30 forks source link

ARM M1 issue Incompatible architecture: Cannot find arm64e in fat binary #136

Open bezromval opened 3 years ago

bezromval commented 3 years ago

The installer report is:

Error Domain=KMErrorDomain Code=3 "Error occurred unloading extensions: Missing extension with identifier com.emu.driver.EMUUSBAudio : Incompatible architecture: Cannot find arm64e in fat binary
Incompatible architecture: Binary is for x86_64, but needed arch arm64e" UserInfo={NSLocalizedDescription=Error occurred unloading extensions: Missing extension with identifier com.emu.driver.EMUUSBAudio : Incompatible architecture: Cannot find arm64e in fat binary
Incompatible architecture: Binary is for x86_64, but needed arch arm64e}
Error Domain=KMErrorDomain Code=71 "Incompatible architecture: Binary is for x86_64, but needed arch arm64e" UserInfo={NSLocalizedDescription=Incompatible architecture: Binary is for x86_64, but needed arch arm64e}
Waiting for the system to finish installation. Can take several minutes.
Executing: /usr/bin/kmutil install --volume-root / --check-rebuild
Rebuilding the kernel
Driver is installed and ready for use!
You can close this window.
Saving session...
...copying shared history...
...saving history...truncating history files...
...completed.
Wouter1 commented 3 years ago

@bezromval thanks for reporting. Yes I was expecting it.

Your report "end of era ..." introduction is not so to the point and a bit negative so I removed that.

I already tried to recompile the code but it does not seem to generate the ARM binary and spits errors where the code wes compiling in the previous XCode And I don't have access to any ARM machine so even if it would compile I would be unable to test this

In short it may take some time before I can fix this

Wouter1 commented 3 years ago

------- notes for myself

Main error message I get from the compiler:

TARGET_OS_OSX not defined
cdefs.h
In file included from ...EMUUSUAudio_info.c

Actually I'm i porting xmach/mach_types.h there. Strange, error comes not from my code but from a standard library.

Wouter1 commented 3 years ago

XCode says the file that causes the error is EMUUSBAudio_info.c Ny project does not have such a file? It does have a file EMUUSBAudio-info.plist. Is this something odd being generated wrong?

I also can NOT edit this 'file'. The filename is also in italics, not clear what's going on

The error says something like "in file included from /Users/wouter/Library/Developer/Xcode/DerivedData/EMUUSBAudio-fjaszixeghwepcdugnckcrbgnuhj/Build/Intermediates.noindex/EMUISBAudio.build/Debug/........

That's not where I put my source files either

Wouter1 commented 3 years ago

Similar error to here https://github.com/catchorg/Catch2/issues/2139 But no solution reported there

Wouter1 commented 3 years ago

https://stackoverflow.com/questions/12132933/preprocessor-macro-for-os-x-targets

gives more details but it's quite confusing

Wouter1 commented 3 years ago

INFOPLIST_PREPROCESS=YES is in the project settings (project.pbxproj). I think this creates the above mentioned file that includes cdef.h

If I uncheck that, the error remains because cdefs.h is still imported through

EMU-driver/USH.h -> IOKit/usb/StandardUSB.h -> IOKit->IOTypes.h -> IOKit/system.h -> sys/cdefs.h

Wouter1 commented 3 years ago

If I put

include

right before the #include<IOKit/usb/StandardUSB.h>

then I get

IOKit/usb/IOUSBLog.h file not found.

Maybe this is a step in the right direction

Wouter1 commented 3 years ago

After just removing that import, I'm back to the TERGET_OS_OSX not defined

But now through the import chain

EMUUSBAudio_info.c -> mach/mach_types.h->stdint.h -> machine/types.h -> i396/types.h -> sys/cdefs.h

Wouter1 commented 3 years ago

Again that is NOT my file. I have turned off the preprocess info.plist file. Also the "Create Info.plist section in binary" is off.

Wouter1 commented 3 years ago

This Derived Data seems wrong. Maybe I can find more about derivedData

https://vojtastavik.com/2018/09/02/what-is-inside-derived-data-xcode/

might help

Wouter1 commented 3 years ago

Is this an XCode configuration issue?

https://developer.apple.com/library/archive/documentation/DeveloperTools/Conceptual/cross_development/Configuring/configuring.html

Wouter1 commented 3 years ago

Well that first line there is already confusing. " you install Xcode, the installer creates a /Developer/SDKs directory"

I HAVE Xcode installed. But I don't have /Developer.

EDIT: but XCode works fine, i now compiled the driver. That first line must be incorrect.

Wouter1 commented 3 years ago

The error disappears if I set in project the Apple Clang-Preprocessing / Preprocessor macros add TARGET_OS_OSX=1

EDIT: This seems indeed needed. When I remove it the build fails.

But then a new error appears in EMUUSBAudioDevice.h line 249 Invalid operands to binary expression ('nullptr_t' and 'UInt8')

Wouter1 commented 3 years ago

If I change that line the NULL to 0 I get another error

IOKit/usb/IOUSBLog.h' file not found

Wouter1 commented 3 years ago

https://developer.apple.com/forums/thread/72248 These files do not exist apparently above 10.10

Wouter1 commented 3 years ago

If I comment out that line, I get another error, EMUUSBAudioDevice.cpp ine 335

because audioEngine.init(NULL) apparently returns boolean. I was comparing it with NULL==. Changed to !audioEngine.init(NULL).

After that, build succeeds! But not clear yet what that means, do we now have fat binary and how to test it

Wouter1 commented 3 years ago

In that building project, I now have "macOS Deployment Target" set to MacOS 10.12" and Base SDK macOS. But it also works if I set the deployment target to macOS 11.0.

Project format is XCode 3.2-compatible.

bezromval commented 3 years ago

I didn't understand any word you write up here, but I'm waiting here for some beta driver for M1 and would like to test it on my MacBook Air with m1 chip and I already have proper usb type c printer cable for it. Just let me know here.

Wouter1 commented 3 years ago

@bezromval thanks for offering help on testing. I have no ARM machine so I can not test anything.

I have at this point something compiling on xcode 12.5 beta. FAIK this should support ARM compilation.

But It's not clear what the compiled driver is at this point. But given that it's 458kB size, strongly suggests it's not fat but just 1 architecture. That probably is intel as I could nowhere select any ARM compilation target...

The notes above are tech notes for myself, documenting what I do. I have not much time so occasionally I have a few hours. It's then essential that I can continue on, rather than redo, what I did last time. Hence all these notes.

Wouter1 commented 3 years ago

Maybe here https://www.lewuathe.com/how-to-make-arm-cross-compilation-on-macos.html or https://www.starmessagesoftware.com/blog/compile-macos-desktop-application-arm64-apple-silicon-m1-chip-cpu

Wouter1 commented 3 years ago

Yes, lipo -info .....kext/Contents/MacOS/EMUUSBAudio gives only x86_64 so not good yet.

Wouter1 commented 3 years ago

The required flags are in the project, under build settings/architecture.

The Architectures are set to Standard Arcitectures (Apple Silicon, Intel) so that looks good.

If I set "Build Active Architectures Only" to "Yes" it builds fine. I had that setting for the Debug mode; and aparently Build builds the debug version...

But if I set it to No then I get another error

EMUUSBAudioEngine line 47 Allocating an object of abstract class type 'EMUUSBAudioEngine (?? looks like a conclusion, not an error message)

Wouter1 commented 3 years ago

Maybe the class EMUUSBAudioEngine does not implement one of the many virtual functios of the superclass?

I see differences, for instance performFormatChange/4 while it was /3 getStatusDescriptor/1 I don't see it in imple getNearestStartTime I don't see it in imple

How does this work, why does it compile for debug but not for run?

Wouter1 commented 3 years ago

Maybe it compiles ok on intel but not on arm? If you switch the "build active architecture only' to 'yes' for debug mode, and if build by default only copmpiles the debug version?

Does that imply that the superclass IOAudioEngine differs between ARM and Intel?

Wouter1 commented 3 years ago

Even weirder, the error ALSO shows in intel. However in spite of the error, the build succeeds there anyway.

In the older compiler, there was no error at all at this place.

If you look at the virtual function performFormatChange, it was always performFormatChange/4. Yet my implementation is performFormatChange/3. How can a function call with an incorrect number of args be accepted at all? The code does not say it was overriding though. What is going on here? Was the function never assumed to override/implement the virtual function? Was it never called?

Wouter1 commented 3 years ago

There is more detail on the error message in the issue navigator on the left. Unfortunately it's very hard to find that detail as there is a huge list of warnings, it's completely hidden between those.

it says

unimplemented virtual method driverDesiredHiResSampleIntervals in EMUUSBAudioEngine

Wouter1 commented 3 years ago

The doc actually says "this is a new method which only exists on ARM platforms and therefore breaks binary compatibility if this is compiled on intel platforms"..... Well at least they have documented this.

Wouter1 commented 3 years ago

Added the missing function, only for ARM. It now compiles. Created new M1 branch and pushed the new code there, including the new kext version v15

Wouter1 commented 3 years ago

@bezromval I just created a first version of the M1 driver. I have no idea what it will do, so please be careful. Maybe run from a separate partition just in case.

There is a new M1 branch containing a new version of the driver : v15

https://github.com/Wouter1/EMU-driver/tree/M1

It's in the v15 directory.

The installation script is NOT recognising that new file yet.

Can you install the v15 driver manually in /Library/Extensions and see what it does?

bezromval commented 3 years ago

Thanks for spending your free time for this! I’ve just tried to install driver. What I've done: 1 boot in recovery mode, lunch terminal and use command csrutil disable. It disabled successfully. Reboot Mac. 2 copy new EMUUSBAudio.kext from v15 to the /Library/Extensions 3 In the system preferences/security and privacy allow this action and system cache was rebuild automatically. 4 Reboot system 5 Sound card did not appeared in system preferences/sound.

Wouter1 commented 3 years ago

@bezromval can you please report any error messages and relevant console reports? I need to know what is going wrong

bezromval commented 3 years ago

Since I was doing it manually there is no error reports. What exactly I should to do to get report in the end?

bezromval commented 3 years ago

Now I've tried: sudo kextcache -I / in terminal and here is the message

Executing: /usr/bin/kmutil install --volume-root / --check-rebuild Not rebuilding KC for arch: 'arm64e' kmutil done

Wouter1 commented 3 years ago

Just checking, you got a prompt after copying the kext into /Library/Extensions? What did it say?

thanks for checking the message.

" Not rebuilding KC for arch: 'arm64e'" mmm not very informative.

regarding the log

https://github.com/Wouter1/EMU-driver/blob/master/FAQ.md and check "How do I make a system log of a mal-functioning driver?".

Wouter1 commented 3 years ago

Trying what it does on the test laptop.

It seems it only works correctly if I do sudo cp -R EMUUSBAudio.kext /Library/Extensions/EMUUSBAudio.kext

I then get a prompt "System extension updated. You must approve it...". Go and approve it in security panel. It requests reboot.

Wouter1 commented 3 years ago

And itl works on my test laptop!

Wouter1 commented 3 years ago

@bezromval maybe something went wrong when you copied it to /Library/Extensions.

Did you use sudo cp -R EMUUSBAudio.kext /Library/Extensions/EMUUSBAudio.kext?

Did you get the prompt "System extension updated. You must approve it"?

Did you approve the kext ?

Did you reboot after approving?

bezromval commented 3 years ago

Using sudo cp -R EMUUSBAudio.kext /Library/Extensions/EMUUSBAudio.kext EMUUSBAudio.kext is in extensions folder after that. There is NO any prompt System extension updated or something. But Iam rebooting anywhere. There is no sound card appears in settings. Prompt appears only if I delete kext from folder. Ask me reboot and is rebuilding cache. Are you sure You've put the right kext in v15 folder? I don't know what to think and what to do. Iam taking a pause for a while.

Wouter1 commented 3 years ago

@bezromval it's not just copying. All permissions have to be right. Only if it's all right you get the prompt. Rebooting is no use until you get that prompt. It means it's not accepted if you don't get the prompt.

It's strange that a prompt appears if you remove. But I did not try.

Of course v15. This is the new folder. I did not change the old folders.

What does ls -al /Library/Extensions show after the sudo cp?

What does sudo kextload /Library/Extensions/EMUUSBAudio.kext show?

Wouter1 commented 3 years ago

@bezromval you can also try sudo kextutil -t /Library/Extensions/EMUUSBAudio.kext what does it show?

bezromval commented 3 years ago

Last login: Wed Mar 17 17:50:26 on console bezromval@MacBook-Air v15 % csrutil status System Integrity Protection status: disabled. bezromval@MacBook-Air v15 % sudo cp -R EMUUSBAudio.kext /Library/Extensions/EMUUSBAudio.kext Password: bezromval@MacBook-Air v15 % ls -al /Library/Extensions total 0 drwxr-xr-x 9 root wheel 288 17 мар 17:51 . drwxr-xr-x 65 root wheel 2080 10 мар 19:49 .. drwxr-xr-x@ 3 root wheel 96 17 мар 17:51 EMUUSBAudio.kext drwxr-xr-x 3 root wheel 96 1 янв 2020 HighPointIOP.kext drwxr-xr-x 3 root wheel 96 1 янв 2020 HighPointRR.kext drwxr-xr-x 3 root wheel 96 23 янв 02:50 SATSMARTDriver.kext drwxr-xr-x 3 root wheel 96 11 сен 2019 SATSMARTLib.plugin drwxr-xr-x 3 root wheel 96 6 дек 2017 SONYDeviceType04.kext drwxr-xr-x 3 root wheel 96 1 янв 2020 SoftRAID.kext bezromval@MacBook-Air v15 % sudo kextload /Library/Extensions/EMUUSBAudio.kext Executing: /usr/bin/kmutil load -p /Library/Extensions/EMUUSBAudio.kext Error Domain=KMErrorDomain Code=31 "Error occurred while building a collection: 1: One or more binaries has an error which prevented linking. See other errors. 2: Could not use 'com.emu.driver.EMUUSBAudio' because: Failed to bind 'ZN8IOSyncer6createEb' in 'com.emu.driver.EMUUSBAudio' (at offset 0x1e0 in DATA_CONST, got) as could not find a kext which exports this symbol com.emu.driver.EMUUSBAudio specific: 1: Failed to bind 'ZN8IOSyncer6createEb' in 'com.emu.driver.EMUUSBAudio' (at offset 0x1e0 in DATA_CONST, got) as could not find a kext which exports this symbol " UserInfo={NSLocalizedDescription=Error occurred while building a collection: 1: One or more binaries has an error which prevented linking. See other errors. 2: Could not use 'com.emu.driver.EMUUSBAudio' because: Failed to bind 'ZN8IOSyncer6createEb' in 'com.emu.driver.EMUUSBAudio' (at offset 0x1e0 in __DATA_CONST, got) as could not find a kext which exports this symbol com.emu.driver.EMUUSBAudio specific: 1: Failed to bind 'ZN8IOSyncer6createEb' in 'com.emu.driver.EMUUSBAudio' (at offset 0x1e0 in DATA_CONST, got) as could not find a kext which exports this symbol } bezromval@MacBook-Air v15 % sudo kextutil -t /Library/Extensions/EMUUSBAudio.kext Executing: /usr/bin/kmutil load --bundle-path /Library/Extensions/EMUUSBAudio.kext Error Domain=KMErrorDomain Code=31 "Error occurred while building a collection: 1: One or more binaries has an error which prevented linking. See other errors. 2: Could not use 'com.emu.driver.EMUUSBAudio' because: Failed to bind 'ZN8IOSyncer6createEb' in 'com.emu.driver.EMUUSBAudio' (at offset 0x1e0 in DATA_CONST, got) as could not find a kext which exports this symbol com.emu.driver.EMUUSBAudio specific: 1: Failed to bind 'ZN8IOSyncer6createEb' in 'com.emu.driver.EMUUSBAudio' (at offset 0x1e0 in __DATA_CONST, got) as could not find a kext which exports this symbol " UserInfo={NSLocalizedDescription=Error occurred while building a collection: 1: One or more binaries has an error which prevented linking. See other errors. 2: Could not use 'com.emu.driver.EMUUSBAudio' because: Failed to bind 'ZN8IOSyncer6createEb' in 'com.emu.driver.EMUUSBAudio' (at offset 0x1e0 in DATA_CONST, got) as could not find a kext which exports this symbol com.emu.driver.EMUUSBAudio specific: 1: Failed to bind 'ZN8IOSyncer6createEb' in 'com.emu.driver.EMUUSBAudio' (at offset 0x1e0 in __DATA_CONST, __got) as could not find a kext which exports this symbol } bezromval@MacBook-Air v15 %

bezromval commented 3 years ago

Hope, you know what all this stuff means because I'm totally not.

Wouter1 commented 3 years ago

@bezromval thanks! It looks like some libraries, or specific versions of libraries, do not exist anymore

Strange, they do exist apparently on the intel version of Big Sur, but not on the M1 version! Just to be sure, you are also on Big Sur?

bezromval commented 3 years ago

Iam on Big Sur 11.2.3. Its the only version can be installed on m1.

philippgadow commented 3 years ago

Hi! Thank you for providing the driver which allowed me to use the EMU tracker pre interface until today. I am very happy that it exist and you keep updating it.

On my work laptop, which is a M1 MBP with BigSur 11.1 (20C69), I can confirm that I get exactly the same errors as bezromval.

Wouter1 commented 3 years ago

The main use is inside EMUUSBAudioEngine start() An IOSyncer called mSyncer is created.

The function of this is not clear to me. IOSyncer seems to have been deprecated already long ago. I can't find any documentation about it. It seems to be used to sync the registration of some IOServiceNotificationHandler. Which is then immediately removed as well. There is a call to mSyncer.signal in the audioDevicePublished method, after found the audio engine to be used. mSyncer.wait could be where it waits for signal.

Maybe this was a mechanism to get the start() to wait for the audioDevicePublished event?

Wouter1 commented 3 years ago

https://github.com/RehabMan/OS-X-Voodoo-PS2-Controller/commit/b33b10c0a6df2e4474d3acc057a6a4f602bf79a1

" //TODO: IOSyncer is deprecated in 10.7 headers, gone from 10.8 headers"

Wouter1 commented 3 years ago

But IOSyncer,cpp is in my source tree. Why is the symbol not resolved?

Wouter1 commented 3 years ago

The cpp code was not part of the project. Apparently I once removed it because it was not needed (but hept the sources where they were)

Wouter1 commented 3 years ago

Tried to add them. But now the driver does not load

I first get OSMetaClass: Kext com.emu.driver.EMUUSBAudio class IOSyncer is a duplicate; kext __ kernel__ already has a class by that name.
kext ... did not start (return code 0xdc00400a
failed to load 0xdc008017
Couldn't alloc class "EMUUSBAudioDevice"