Can't configure (pre-compile) on Apple Silicon BigSur

[Shambler2]

Hi,

Quite possibly doing something wrong here - first time attempting to compile something on this machine.

Followed the BigSur instructions, used the ARM release of homebrew to install dependencies, including the package provided by vashpan in the discussions (cdalvaro/tap/wxmac)

Currently running BigSur 11.2.3

Ran the configure command and got the following output:

user@MacBook-Pro PCemV17macOS-main % ./configure --enable-release-build
checking build system type... arm-apple-darwin20.3.0
checking host system type... arm-apple-darwin20.3.0
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... ./install-sh -c -d
checking for gawk... no
checking for mawk... no
checking for nawk... no
checking for awk... awk
checking whether make sets $(MAKE)... yes
checking whether make supports nested variables... yes
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables... 
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking whether gcc understands -c and -o together... yes
checking for style of include used by make... GNU
checking dependency style of gcc... gcc3
checking for g++... g++
checking whether we are using the GNU C++ compiler... yes
checking whether g++ accepts -g... yes
checking dependency style of g++... gcc3
checking whether to build for release... yes
checking whether to enable debugging... no
checking whether to enable networking... no
checking whether to use ALSA for MIDI output... no
checking for cpu... configure: error: Unsupported CPU. arm
user@MacBook-Pro PCemV17macOS-main % make
make: *** No targets specified and no makefile found.  Stop.

Seems like it's not matching the CPU to one of the ones defined in the configure script and just failing from there? Not familiar enough with configure scripts to troubleshoot myself yet though will keep fiddling

[almeath]

Thanks for the feedback. Unfortunately, I do not yet have access to an M1 Mac, so it is difficult for me to trouble-shoot this issue. The other important thing to note is that the version of PCem that we provide here is a patched version of the official release, and the v17 release does not officially support ARM architectures. So in your situation it will be compiled for Intel to run in Rosetta 2.

I know from reading the PCem forum that experimental ARM builds are circulating, but again I have no way to test them, and I do not know if ARM support will automatically translate to native Apple Silicon support.

I will see if @kyr0 has any suggestions and/or access to an M1 Mac.

[Shambler2]

Thanks for the response! I wasn't aware there were arm specific builds running around, I haven't been able to locate any as yet.

I did manage to hack my way into getting it compiling, however ran immediately into a SEGFAULT once running it - not super surprised if there's actually no ARM support in this build!

Happy to test anything at a later date if something is worked out

[almeath]

No problem, it is good to know that you at least got past the compilation issue. I have sent a query to our Github team and will keep you updated. In relation to ARM, I have not actually seen evidence of the builds, just conversations referring to their existence. If you are not already a member of the PCem forum you might want to register there. Just note, it can take a while to get an "acceptance" from the moderator. Make sure your email address does not look anything like a spammed address.

[kyr0]

Heyho,

my understanding of the situation is, that you might get a build working itself, but the recompiler/codegen in PCem will generate x86_64 instructions only. The original author of PCem probably never intended it to run on ARM natively thus the support for ARM would also need to include a larger patchset for PCem internals code :)

However - did you try to download my binary build (u can find it in the release link) and execute it via Rosetta? Does this work at least?

[Shambler2]

Hi, I did actually (first thing I tried) - ran into an issue missing libpng. Installed the ARM version of the library but that obviously wasn't going to work :P Didn't want to go through the hassle of installing the x86 version of homebrew so tried to see if I could compile an ARM version first.

I actually don't have a x86/64 Mac so wouldn't be able to compile that version of the app - but if that missing library could be sorted out I imagine it would run under Rosetta (most other software does out of the box anyway)

[almeath]

I extracted my libpng install from usr/local/lib and usr/local/Cellar on my Intel Mac. You could try placing the files in those locations on your system and attempting to run.

libpng_mojave.zip

I could not extract the files from a Big Sur install, as Apple introduced a dynamic linker cache for all system supplied libraries, and it seems there are only "alias" files under usr/lib (with no sign of where the originals are hidden .. no doubt somewhere on the sealed system volume).

https://developer.apple.com/forums/thread/655588

[Shambler2]

Hiya, thanks! I had to play around a bit as the path for the library didn't match, but moved the files around into the expected path and it progressed further - to the next library that was also missing :P

From the crash report: `System Integrity Protection: enabled

Crashed Thread: 0

Exception Type: EXC_CRASH (SIGABRT) Exception Codes: 0x0000000000000000, 0x0000000000000000 Exception Note: EXC_CORPSE_NOTIFY

Termination Reason: DYLD, [0x1] Library missing

Application Specific Information: dyld: launch, loading dependent libraries

Dyld Error Message: dyld: Using shared cache: 1C99695B-39A1-3CF1-BCD1-AB8BF015FD98 Library not loaded: /usr/local/opt/jpeg/lib/libjpeg.9.dylib Referenced from: /Applications/PCem.app/Contents/MacOS/libwx_osx_cocoau_xrc-3.0.dylib Reason: image not found ` Had a poke around in the packaged app and it doesn't provide libpng or libjpeg (or presumably some others?) Not familiar with packaging up of Mac apps (though pretty familiar with Windows) so not sure if these can be imported/included?

EDIT: Can manually add files to the package but it doesn't seem to detect them - wants the file to exist where specified

[almeath]

Ok, here is libjpeg so you can test further.

libjpeg_mojave.zip

I will look into the library packaging option, as I took a similar approach with my DOSBox fork.

[Shambler2]

Thanks for that! Sorry it took a while to swing back around. It has now progressed to the next dependency (libtiff).

Not sure how many more there are but happy to keep testing if you've got them available.

Packaging the libraries along with the executable is probably the best way - not sure if there's any changes needed at compile time to reference the libraries locally?

[mikoenig]

The original author of PCem probably never intended it to run on ARM natively thus the support for ARM would also need to include a larger patchset for PCem internals code :)

PCem actually already includes a dynarec for ARM (codegen_backendarm64*.c) and I think Sarah mentioned that she tested it on a Raspberry Pie. It seems to work when I run PCem on a virtualised Ubuntu for ARM64 on my M1. Otherwise an emulated Pentium 233 MMX probably wouldn't reach 100%, I guess. Unfortunately I have problems with the mouse pointer disappearing in PCem under Ubuntu, so I cannot really test past the Windows installation.

So the code is there, but the build process would have to be persuaded to use the ARM64 dynarec when built for/on an Apple Silicon processor. My first hurdle in that case was that the processor architecture is simply reported as "arm", while the configure script seems to expect "arm64". I tried to edit "configure.ac", but opened a can of worms in terms of additional tools that I suddenly needed. Also I have absolutely no experience with creating/editing configure scripts, so I decided to skip my tests with this software and continue testing other software on the new hardware.

[almeath]

This is the full list of dependencies, which I have bundled into my PCem app using dylib bundler:

Here is libtiff:

libtiff.zip

[almeath]

I have my own custom app bundle that is self-contained. I used dylib bundler to place all the dependencies inside the app bundle. If you would like to test it, it is available here:

https://www.dropbox.com/s/0opc7h2gcui10yo/PCem%20app%20bundle.dmg.zip?dl=0

It is based on the same process I used to package up my custom DOSBox app.

If you take a look inside Contents/Resources it should be fairly self-explanatory how it works. You can change the config file to your liking, but I set it to launch into full screen if you just open the app and then click anywhere in the window. You might not like the shader presets but they are intended for my 5k screen. You can disable those too. Oh, and my custom resolution is set to 5k too .. when you launch, immediately right-click on the screen and go into settings to alter it to your native screen resolution. Alternatively, just edit the config file manually inside the wrapper, before you launch the app.

Just remember that due to the current bug in the mac port of PCem, you are best off using option-command-esc to force quit the app when you shut down Windows. If you change machine settings while running the app, you need to do the normal shutdown process of switching back to window mode, right-clicking on the screen to get the config menus, then use the shutdown command from there (and get the expected crash on the way out). If you do not do that, it will not write the changes to the config file. Once you have your preferred config in place, it does not seem to do any harm to use force quit. One day I would like to solve that bug..

This is obviously an Intel build so if it runs at all on the M1 it will be under Rosetta 2. Of course, if you are able to compile your own binary with the library files I provided above, then you could try replacing the launch binary in Contents/Resources/pcem

[Shambler2]

The original author of PCem probably never intended it to run on ARM natively thus the support for ARM would also need to include a larger patchset for PCem internals code :)

PCem actually already includes a dynarec for ARM (codegen_backendarm64*.c) and I think Sarah mentioned that she tested it on a Raspberry Pie. It seems to work when I run PCem on a virtualised Ubuntu for ARM64 on my M1. Otherwise an emulated Pentium 233 MMX probably wouldn't reach 100%, I guess. Unfortunately I have problems with the mouse pointer disappearing in PCem under Ubuntu, so I cannot really test past the Windows installation.

So the code is there, but the build process would have to be persuaded to use the ARM64 dynarec when built for/on an Apple Silicon processor. My first hurdle in that case was that the processor architecture is simply reported as "arm", while the configure script seems to expect "arm64". I tried to edit "configure.ac", but opened a can of worms in terms of additional tools that I suddenly needed. Also I have absolutely no experience with creating/editing configure scripts, so I decided to skip my tests with this software and continue testing other software on the new hardware.

This is what I did originally - I ended up doing some horrible things to the build script to force it to build for arm64 - but then ran into more dependency issues (SDL2 mainly). Hacked away at it a bit more and was able to build but immediately segfaulted.

[Shambler2]

I have my own custom app bundle that is self-contained. I used dylib bundler to place all the dependencies inside the app bundle. If you would like to test it, it is available here:

https://www.dropbox.com/s/0opc7h2gcui10yo/PCem%20app%20bundle.dmg.zip?dl=0

It is based on the same process I used to package up my custom DOSBox app.

If you take a look inside Contents/Resources it should be fairly self-explanatory how it works. You can change the config file to your liking, but I set it to launch into full screen if you just open the app and then click anywhere in the window. You might not like the shader presets but they are intended for my 5k screen. You can disable those too. Oh, and my custom resolution is set to 5k too .. when you launch, immediately right-click on the screen and go into settings to alter it to your native screen resolution. Alternatively, just edit the config file manually inside the wrapper, before you launch the app.

Just remember that due to the current bug in the mac port of PCem, you are best off using option-command-esc to force quit the app when you shut down Windows. If you change machine settings while running the app, you need to do the normal shutdown process of switching back to window mode, right-clicking on the screen to get the config menus, then use the shutdown command from there (and get the expected crash on the way out). If you do not do that, it will not write the changes to the config file. Once you have your preferred config in place, it does not seem to do any harm to use force quit. One day I would like to solve that bug..

This is obviously an Intel build so if it runs at all on the M1 it will be under Rosetta 2. Of course, if you are able to compile your own binary with the library files I provided above, then you could try replacing the launch binary in Contents/Resources/pcem

I'll give that build a go and let you know tomorrow how it went - hoping that in the short term Rosetta performs well 'enough' until a native ARM build can be worked out

[almeath]

No problem, take your time with it. I wish I could help out more on the ARM issue but I am not sure when I will get my first Apple Silicon machine.

[Shambler2]

Yeah just did a quick test then - got past that bit but now failing on ANOTHER library - gave up and installing homebrew to run under Rosetta now. At least that way I can install the dependencies myself without having to keep bothering you each time ;)

The dropbox build unfortunately bombed immediately - macOS decided it was corrupt and refused to run it after that

[Shambler2]

Alright I think I've been able to confirm that PCem won't work under Rosetta (for now) - dependencies all fulfilled directly by homebrew packages (x64 versions) except had to compile wxWidgets to get the version hard linked in the packaged up version.

Definitely gets much further on (Rosetta actually kicks in now) and the app appears to start launching - however crashes before any UI appears.

The error I'm getting now is a Rosetta 2 specific error:

Crashed Thread:        1  com.apple.rosetta.exceptionserver

Exception Type:        EXC_BREAKPOINT (SIGTRAP)
Exception Codes:       EXC_ARM_BREAKPOINT at 0x00007ffdfffbe36c (brk 1)
Exception Note:        EXC_CORPSE_NOTIFY

Termination Signal:    Trace/BPT trap: 5
Termination Reason:    Namespace SIGNAL, Code 0x5
Terminating Process:   exc handler [52216]

Application Specific Information:
rosetta error: unexpectedly need to EmulateForward on a synchronous exception x86_rip=0x7fff201b1df5 arm_pc=0x7ffe964bf720 num_insts=5 inst_index=4 x86 instruction bytes: 0x56415741e5894855 0x8941505354415541

I imagine at this point a Rosetta specific fix will need to be implemented, or the ARM dynamic recompiler as mentioned above needs to be ported to the macOS platform

Thanks so much for all your help to this point! Wish I could perform more of the dev work myself but I'm mostly new to macOS - still wrapping my head around the architecture as well

[mikoenig]

Alright I think I've been able to confirm that PCem won't work under Rosetta (for now)

While unfortunate, I think the best option would still be trying to get it to run native. As surprised as I am that some emulators with dynarecs run at a decent speed with Rosetta 2, the doubled binary translation can never be good for the overall performance.

I imagine at this point a Rosetta specific fix will need to be implemented, or the ARM dynamic recompiler as mentioned above needs to be ported to the macOS platform

I‘m not totally sure how stable the ARM64 code generation is. Windows 2000 blue-screened during installation, while Windows 98 managed to get to the configuration of time zones, when the disappearing mouse pointer tripped me up (the blue screen could have been a different issue though).

Dynamic code generation seems to be a bit of a hurdle on Apple Silicon that Apple released some of information on it: https://developer.apple.com/documentation/apple-silicon/porting-just-in-time-compilers-to-apple-silicon

I believe the maintainers of UTM also had some problem with JIT code generation, although I‘m not sure if everything was covered in the link mentioned above or not.

Thanks to anyone, who spends their private time to get PCem running on macOS! With x86 virtualization out of the picture it would be a good option to run some of the more legacy code.

[almeath]

I know that Apple Silicon is causing similar issues for DOSBox. To quote Dominus on Vogons:

“ so far there is no satisfying solution for the silicon. The dynamic core is not working (crashes) and there are solutions in other forks but these are mostly hackish and affects other architectures negatively (e.g. works on silicon but crashes on x86_64). BUT so far Rosetta2 is amazing and SVN runs really fast.”

It may be that Rosetta 2 is the only viable solution in the short term. Proper Apple Silicon support is not just a matter of ARM compatible builds .. it sounds like there are many optimizations required, perhaps requiring very deep rewrites of the underlying code.

I suspected this was going to happen with anything related to x86 emulation , and that is why I chose to purchase a maxed out Intel Mac and try to ride out the transition period. I will still get an Apple Silicon machine as a secondary system, perhaps the 2nd or 3rd generation, so that I can teach myself the ins and outs of the platform.

[ghost]

I was able to build PCem for use with Apple Silicon Macs back in December, shortly after they came out and v17 was released. That honestly was the easy part. The hard part is running PCem - low level debugging refers to block issues. Most likely due to Apple's W^X permission requirements. I haven't bothered to work them out as I've been focusing on Dolphin and MelonDS instead.

@almeath , if you'd like to take a shot at it I can work with you to give you remote access to my M1 MBP when we are both available. Also, any chance of a change so that the PCem folders are saved under Application Support instead of the main user folder?

PS: @Shambler, I was able to get PCem to work under rosetta 2 using the directions for Big Sur on the main page. As much as you will probably not enjoy it, your best bet would be to start over from scratch.

[almeath]

@almeath , if you'd like to take a shot at it I can work with you to give you remote access to my M1 MBP when we are both available. Also, any chance of a change so that the PCem folders are saved under Application Support instead of the main user folder?

I made the change to place config files in ~/Library/Application Support/PCem/ as I agree that makes more sense for a macOS app, and avoids cluttering up the home folder.

Thanks for the offer - I will take a rain check on that for now. I am in Australia, so the timezone difference will likely be difficult, and my work/life balance is not great right now either. Before I could tackle Apple Silicon, I would like to try to resolve a couple of nagging problems with the current Mac build, such as the crash on exit.

[ghost]

if you'd like to take a shot at it I can work with you to give you remote access to my M1 MBP when we are both available. Also, any chance of a change so that the PCem folders are saved under Application Support instead of the main user folder?

I made the change to place config files in ~/Library/Application Support/PCem/ as I agree that makes more sense for a macOS app, and avoids cluttering up the home folder.

@almeath It took me forever to realize this version of PCem was actually respecting the Linux use of $home. I was used to keeping all my stuff in the same directory as the executable.

Did you mean to get rid of the pcem.cfg file too or was that just coincidence?

Thanks for the offer - I will take a rain check on that for now. I am in Australia, so the timezone difference will likely be difficult, and my work/life balance is not great right now either. Before I could tackle Apple Silicon, I would like to try to resolve a couple of nagging problems with the current Mac build, such as the crash on exit.

I know what you mean. It was always fun trying to coordinate with Waluigi for melonDS and they are in the UK. I should be asleep right now - it was just sheer chance that I am still up at the moment.

[ghost]

Surprise! PCem is now confirmed to compile and run natively on arm Macs. And even better, the standard homebrew install of wxmac works oob.

[Shambler2]

Surprise! PCem is now confirmed to compile and run natively on arm Macs. And even better, the standard homebrew install of wxmac works.

Interesting! Was that just the changes made to support the Mac config directory? Or a private branch somewhere?

[ghost]

Different branch. I was about halfway there myself when I found out - so I tossed mine in the trash, figured out a couple of bugs remaining in their code and viola.

Performance for a pentium II 450 has the potential to be as smooth as butter in native apple silicon.

[almeath]

Can you share the link to this branch? I cannot find it with Google search.

[almeath]

I'm not that impressed with those new iMacs. I'll reserve judgment for the successor to the 27 inch models, which would hopefully be more "pro" focused.

[ghost]

Can you share the link to this branch? I cannot find it with Google search.

Kode54 is the creator and it is more a code patch than a branch.

This is not an official benchmark or anything, but here is a basic comparison of x86_64 PCem and Apple Silicon native PCem. Virtual PC specs: PII/450, 128MB RAM, 2GB HD, Voodoo 3 3000, Soundblaster AWE32 and Windows 98 SE. Host Mac specs: M1 MacBook Pro, 16GB RAM, 1TB on board NVME storage.

X86_64: When booting Win98SE, performance is very sluggish and choppy. PCem reports emulation speeds around 26% during boot up. Same low % when using system after boot up has completed, even with Voodoo 3 drivers installed and Dynamic Recomplier grayed out but has a check mark so I assume it is selected (I heard on the forums that DR is broken on Macs), 100% speed when idle.

Apple Silicon Native: When booting Win98SE, performance is very smooth. PCem reports emulation speeds around 90-100% during boot up. Windows startup chime has some minor stutter to it, Welcome to Windows 98 music plays fine. Same level of performance when using system after boot up has completed. Dynamic Recompiler is grayed out but has a check mark, so I assume it is selected.

For some reason the song "Anything you can do, I can do better" popped into my head after seeing this.

@almeath, since I have confirmed that PCem not only is buildable on Arm Macs but runs at a substantially higher level of performance, I recommend closing this issue and reaching out to Kode54 to ask if you can merge their code into this project.

[Shambler2]

That sounds great! My target would probably be a little lower spec anyway so looking forward to decent performance.

I did find the patch file (https://gist.github.com/kode54/eadc26361c090ed48be8faf803cd2313) - I'm assuming this patch file is intended to be applied against this macOS specific branch of PCEM (or is it the main branch?)

@StLouisCPhT you had mentioned there were a few additional bugs that you had fixed up - are they in the patch linked above or are they pretty straight-forward to resolve?

I'll have a go at compiling it on the train home this afternoon

[ghost]

That sounds great! My target would probably be a little lower spec anyway so looking forward to decent performance.

I did find the patch file (https://gist.github.com/kode54/eadc26361c090ed48be8faf803cd2313) - I'm assuming this patch file is intended to be applied against this macOS specific branch of PCEM (or is it the main branch?)

@StLouisCPhT you had mentioned there were a few additional bugs that you had fixed up - are they in the patch linked above or are they pretty straight-forward to resolve?

I'll have a go at compiling it on the train home this afternoon

@Shambler2

I have only tested it with the main branch. Kode54 is in the process of cleaning things up with the code as well as adding in the fixes I had to make, so I would probably wait a couple more days first.

[kyr0]

Hey guys, unfortunately I cannot spend much of my time these days but I'm following this discussion and I'm very happy and excited to see your progress and contributions. I just invited you both to the project so that you can actively and directly work with the code.

In case you guys wanna build new release versions, please find the other repos of mine in this org. I found a nice way to come up with a DMG..

If you guys prefer, you can also change the "home" path for mac from /Users/$user/.pcem - to Application Support. I was just unsure about the usability for novice users. However, if we'd add a lil button in the UI that would allow to open this folder in Finder or just do that when we show the error message that nothing has been found, then this would be a perfect fit I guess -- standards and usability =)

Thanks alot again!

[kyr0]

That's the bundler for creating DMG's. Please note the different arguments for make. They make space to rewrite the paths after linking so that they can be exchanged with the static paths to point inside the app bundle (done via script)

https://github.com/PCemOnMac/bundler

[kyr0]

Pls also welcome @kode54 - I've invited you as well just in case it helps :)

[kode54]

Should work a treat now, with the pull I've just filed. I think.

[kode54]

Spoke too soon, some things are unstable. Maybe there's unexpected issues on Apple Silicon? For instance, Internet Explorer crashes upon loading the startup page, and Final Reality fails to perform its first blit during the benchmark, and already glitches out on startup.

[Shambler2]

Just wanted to close out this issue - with the changes posted over the last few days and merged into main the issue I was having has been resolved.

I've successfully compiled the main branch with only minor issues - will raise a new issue for them but closing this one.

Thanks everyone for your help both stepping me through resolving the initial issue, and ultimately fixing the base functionality on M1 Macs as well!

[ghost]

Almeath, Kode54 and Kyr0 (wherever you are), you might have missed it over on the PCem forums - Sarah asked if there are plans to merge all this back to the mainline.

http://pcem-emulator.co.uk/phpBB3/viewtopic.php?p=15407&sid=19b1cb2d5f9592386218d145552be25a#p15407

[kyr0]

@StLouisCPhT Hi, thanks for the heads up. I didn't miss it -actually we've started a discussion thread 2 weeks ago to discuss this: https://github.com/PCemOnMac/PCemV17macOS/discussions/14 Merging back is of course a lil bit more work because we also need to make sure that our apple platform changes doesn't break the other platforms. Means: A lot of more testing etc. Therefore I started the discussion thread to see how you guys think. Would you have time to help in testing and merging the code? Would also be cool to hear feedback from you @almeath :) Maybe in the discussion thread so we're not going too offtopic :)