Make Boxer 64-bit compatible for macOS 10.14

[simonbrueckner]

A couple of days ago, macOS 10.13.4 was released which starts issuing warnings that apps which don't support 64-bit will be retired with macOS 10.14 (expected in Autumn '18).

This means that Boxer will in its current setup stop working on the latest macOS later this year.

I get it that the main issue is that underlying DosBox is not 64-bit compatible - but at least it has been compiled with 64-bit support a couple of years ago: https://hexeract.wordpress.com/2016/09/10/building-dosbox-as-x64-binary-for-macos-sierra/

It would be great to keep Boxer running for the macOS community!

[aminta]

That also includes support for the shader you previously included 👏🏻

@stuartcarnie Thanks, now all it works! Just a last question: in some games like "Lakers vs. Celtics and the NBA Playoffs" we have to type nba.exe m32 to activate the Roland32. How can I make this in Boxer? I've tried to add it to [autoexec] section of the game .conf but it doesn't work...

[dreamer]

@stuartcarnie

I don't know if it's already too late to consider for your project, but wanted to notify you about dosbox-staging - it might be a better choice for you long-term than forking upstream. I see some prospect for co-operation, as Boxer needs stable emulator backend while dosbox-staging needs macOS testers and developers :)

Some technical notes you might be interested in:

Upgraded Boxer and DOSBox to SDL2 (via C_SDL2 define similar to other forks)

If you followed other forks' route by integrating old SDL2 patch without replacing SDL_sound first, the result is broken CD audio emulation and broken keymapper. We started with the same SDL2 patch, but purged all SDL1.2 code from the codebase, removed all cruft we could find (like Windows 9x or OS/2 support), and fixed a ton of things over last 8 months.

OpenGL is gone

OpenGL was never a performance bottleneck; removing it will make it harder for you to support Voodoo emulation in the future (which is missing from vanilla dosbox right now; We agree, that Metal renderer should be the primary one for the emulator on macOS in the future and we're very interested in supporting Metal as standalone output backend, but as for now we support it only via SDL2 texture. If you don't want any OpenGL code on macOS, you can just disable the appropriate feature. We would be happy to support Metal backend, but want to cleanup sdlmain first (we're still fixing it and are in the process of removing surface output), so we could create a proper interface for writing backends (instead of dumping all the new code in the same file).

Is OpenEmuShaders cross-platform or macOS-only?

[almeath]

Is OpenEmuShaders cross-platform or macOS-only?

I think the OpenEmu shaders are translated from GLSL into a Metal-compatible macOS-only version

[stuartcarnie]

@dreamer

Is OpenEmuShaders cross-platform or macOS-only?

OpenEmuShaders is a Metal implementation to support https://github.com/libretro/slang-shaders, which are cross-platform. I use SPIRV-Cross, glslang and SPIRV-Tools to handle all the automatic translation from Vulkan / glsl to Metal.

[stuartcarnie]

Regarding OpenGL, my primary motivation is to support current and future Apple platforms (Si), and OpenGL is not a viable solution for that.

removing it will make it harder for you to support Voodoo emulation in the future

I took a look at dosbox-x's implementation of the voodoo support, and it is not a trivial mapping to OpenGL (i.e. it isn't a pass thru), so I'd be comfortable implementing a Metal version of that for Boxer. In all likeliness, I'd consider doing it in such a way as to support multiple back-ends so that OpenGL can be used for other platforms.

[dreamer]

@stuartcarnie In dosbox-staging we're discussing Voodoo emulation support here: https://github.com/dosbox-staging/dosbox-staging/issues/339 (TL;DR; no, we won't do passthrough, yes, OpenGL is required to integrate a workable version of existing support of Voodoo emulation). Just wanted to notify you, that there's more work to cleaning up community patches like SDL2 than it looks like on the first sight. We (talking for me and @kcgen here) are not related to DOSBox-X project, although we cooperate a little bit and share knowledge and some fixes with -X maintainers (when applicable).

If you're not interested in cooperating with dosbox-staging, that's fine (I'll keep a tab on what' you're doing similar to other dosbox forks); if you or @MaddTheSane will find it valuable to cooperate on one emulator backend, then ping us via mail or GitHub issue :)

[stuartcarnie]

@MaddTheSane what do you think about using the dosbox-staging backend, I had been thinking about it before @dreamer mentioned it (as I looked at dosbox-x too), but I think dosbox-staging is the right choice.

[MaddTheSane]

Agreed.

[MaddTheSane]

Migration to dosbox-staging is now in my fork: bd94a630

[stuartcarnie]

OMG @MaddTheSane you are awesome!

[stuartcarnie]

I'm keen to add voodoo support with a Metal backend next!

[almeath]

Thanks so much to @stuartcarnie, @MaddTheSane and others for keeping Boxer alive.

I am more than happy to upload pre-built Boxer/Bundler/Standalone as I was doing earlier, for those unable to build it themselves.

However, I cannot get @stuartcarnie's branch to successfully build under Mojave 10.14.6 (I have no intention of "upgrading" to Catalina anytime soon).

I run the following before building:

$ git pull --recurse-submodules

But then I get a tool dependency error in Xcode: "Command ExternalBuildToolExecution failed with a nonzero exit code"

Full error:

I see the part about cmake, but I have already installed it using homebrew.

[aminta]

@almeath here it is a working build.

[almeath]

@almeath here it is a working build.

Thanks! I should really get around to installing Catalina on my secondary system so that I can at least play around with it and test building in that environment.

@aminta, is there any chance you could also build and upload the "Standalone" and "Bundler" apps that come with Boxer?

[chrismaaz]

Just wanted to say thank you to everyone involved in continuing to develop Boxer so we all can enjoy retro gaming on MacOS. Your work is appreciated and we all salute you! SDL2 with Metal and possible Glide support would be a dream for sure.

[chrismaaz]

@almeath here it is a working build.

Is this a recent build? Says december 18, 2019. So I assume it's not built using the latest fork? I tried it and it still runs like tar for me.

[almeath]

@almeath here it is a working build.

Is this a recent build? Says december 18, 2019. So I assume it's not built using the latest fork? I tried it and it still runs like tar for me.

I agree, it looks like an earlier build and the benchmarks are horrible. I am still trying to compile the latest build in Mojave, if I can get around the error I am experiencing in Xcode.

[aminta]

@almeath , @chrismaaz forgive me, it was a not correct build I've just compiled the last version in Catalina, together with Standalone and Bundler, and updated the original download link!

[chrismaaz]

@almeath , @chrismaaz forgive me, it was a not correct build I've just compiled the last version in Catalina, together with Standalone and Bundler, and updated the original download link!

Looking great! Getting equivalent speeds to Dos Box 0.74-3 now <3

[aminta]

That also includes support for the shader you previously included 👏🏻

@stuartcarnie Thanks, now all it works! Just a last question: in some games like "Lakers vs. Celtics and the NBA Playoffs" we have to type nba.exe m32 to activate the Roland32. How can I make this in Boxer? I've tried to add it to [autoexec] section of the game .conf but it doesn't work...

I answered myself: in these cases you have to change directory first, so:

cd nba nba.exe mt32

[blogabe]

@aminta I'm getting a 404 error downloading from https://www.terredainventare.it/dos/BoxerStuart.zip

[aminta]

@aminta I'm getting a 404 error downloading from https://www.terredainventare.it/dos/BoxerStuart.zip

New link is https://www.terredainventare.it/dos/BoxerStuartComplete.zip

[agustincordes]

Wait... so @MaddTheSane's fork is already updated to dosbox-staging and also merged @stuartcarnie's improvements?! This is outstanding!

I'm still looking forward to contribute when I find the time. One feature I had in mind was to allow management of shadow boxed files, ie: options to browse them with a menu shortcut and erase them (that is, reset a boxed game to its original state). That sort of quality-of-life and UI improvements. I'm very happy to see Boxer with renewed life! 🙌

EDIT: I see those features to manage "player data" (shadow boxed files) are already implemented. Funny, I had missed them!

[stuartcarnie]

I've been pondering how dynarecs work in emulators, and many of them grab a block of instructions and convert to native code. DOSBox, for example, consumes 32 instructions from the instruction stream and tries to generate native code:

https://github.com/MaddTheSane/dosbox-staging/blob/b1c063adcf7bc27c09b9b4f65133f337f820918d/src/cpu/core_dynrec.cpp#L238-L239

Seems like implementing a "simple" tracing JIT could yield better results.

tl;dr how effective would a tracing JIT be for optimize hot code (almost always loops). A tracing JIT will profile -> trace -> generate -> execute.

[dreamer]

@stuartcarnie you're pointing to only a single Block, and there are multiple dynamic code blocks precompiled - I can't say for sure how much different this approach is from full-blown JIT (perhaps it's already equivalent, perhaps not).

Anyways, there are some problems in this area - I am currently finishing up small cleanup in order to fix W^X memory access (I don't know if it's an issue on macOS - various BSD platforms have similar protections as Linux+SELinux but I haven't seen it on macOS yet), after that I am planning to implement hugepages support to investigate improvements for cache misses (no idea if it'll improve performance, but I hope so). (I did some performance tuning using Linux perf and concluded, that cache page trashing is currently the worst source of slowdowns in the emulator, but I'm always eager to see what other people's analysis will yield :) ).

Also, right now there are 2 parallel dynrec implementations - faster one for x86/x86_64 machines (core_dyn_x86) and slower one, but supporting multiple architectures (core_dynrec); the default depends on your target architecture (but you can switch via configure option).

[MaddTheSane]

According to Addressing Architectural Differences in Your macOS Code in the section Update Just-In-Time Compilers, it says JIT code needs to be written to "prevent all memory pages from being simultaneously writable and executable." While this won't be an issue for Intel Macs, it will be an issue on ARM Macs.

[stuartcarnie]

@dreamer, Boxer has some hacks and the core_dynrec is working on Si. I plan to make the changes more consistent with the link that @MaddTheSane shared.

[dreamer]

Sorry, I have no idea what Si is :)

core_dynrec in dosbox-staging supports following architectures: ARMv6, ARMv7, ARMv8 (also Aarch64, so I guess future macOS will be covered), PPC, PPC64 (tested as BE on AmigaOS and LE on Linux), x86, and x86_64 (but these two are only for development purposes as far as I can tell - it's better to use core_dyn_x86). There's also MIPS, but nobody tried that in years, so it might be broken.

If you want to discuss W^X work in more detail - maybe I should create a dedicated issue for this topic? So the discussion and ongoing status won't be lost in here ;) - in staging bugtracker we have Linux-specific issue only ATM, so having one covering macOS as well should be beneficial; the code that needs fixing is cross-platform anyway.

[getaaron]

Sorry, I have no idea what Si is :)

Si == Apple Silicon == ARM

[dreamer]

Ah;  does not show up on non-Apple systems, so it's a square on my side ;)

[thawking]

I am useless at programming and thus sadly cannot help at all on this project, but I just want to say a massive "thank you" to everyone who has worked on getting Boxer working under Catalina. It's a great application, and I've really missed it since upgrading to 10.15. You guys are the best! <3

[getaaron]

Anyone have tips for fixing the "session is not ready" error?

[agustincordes]

@stuartcarnie, love what you did with the shader properties panel! It works really well (only the scanlines setting isn't doing anything at the moment).

I know it's an early implementation but the ability to save presets would be a killer feature. Superb job 👏

[stuartcarnie]

@agustincordes good to hear!

The shader properties are parsed from the parameters within the shader files dynamically. We are currently limited to the 3 options exposed by Boxer, so the CRT one is CRT Geom:

https://github.com/MaddTheSane/Boxer/blob/2ff9e5b5905cdad915eec50d7455abde4a60fba2/Boxer/Metal%20Rendering/BXMetalRenderingView.m#L121

It appears that because USEGAUSSIAN is defined:

https://github.com/MaddTheSane/Boxer/blob/2ff9e5b5905cdad915eec50d7455abde4a60fba2/Resources/Shaders/CRT%20Geom/crt-geom/shaders/crt-geom.slang#L76

The scanline parameter is unused:

https://github.com/MaddTheSane/Boxer/blob/2ff9e5b5905cdad915eec50d7455abde4a60fba2/Resources/Shaders/CRT%20Geom/crt-geom/shaders/crt-geom.slang#L183

The multi-pass shader engine is built to support https://github.com/libretro/slang-shaders, but we make them friendlier by wrapping them in packages here:

https://github.com/OpenEmu/slang-shaders/releases/tag/v1.0

Boxer will support these, including saving presets and adding your own custom shaders 🎉

[stuartcarnie]

Incidentally, I want to add more metadata so we can make a better UI than the current shader properties panel.

[agustincordes]

@stuartcarnie, yeah, I noticed the USEGAUSSIAN flag is enabled by default. It wasn't in my take of the shader and I understand it's an outdated (slower) code. Of course, this doesn't matter if you're planning to support custom shaders and presets!

I'm growing fond of Lottes, which I've also tweaked for ScummVM. Glad to see it's in the default pack.

[freecodecampster]

@aminta I'm getting a 404 error downloading from https://www.terredainventare.it/dos/BoxerStuart.zip

New link is https://www.terredainventare.it/dos/BoxerStuartComplete.zip

Thanks, this build is working on Big Sur beta too.

[WizardOfWor1969]

Sound stutters on my MacBook Pro running 10.15.6

[chrismaaz]

Sound stutters on my MacBook Pro running 10.15.6

I've noticed some audio stutter / white noise just when I launch Boxer and close it, but never during gameplay on Catalina.

[WizardOfWor1969]

It's when you move the volume slider , or adjust the window size. I noticed earlier versions muted the sound. I also liked the older versions that let you select the command line instead of having to select and executable or bat file etc.

[stuartcarnie]

The sound will be fixed as we are rearchitecting DOSBox to run on a background thread.

[WizardOfWor1969]

Makes sense, I figured it was a threading problem as it happens when dialogs pop up too. Thanks to everyone who is coding to make this cool app continue on the mac.

[kcgen]

@stuartcarnie , that will do the trick, as SDL2 blocks on some external window-manager events. This also happens in Linux when toggling focus or changing from full to windowed-mode.

we are rearchitecting DOSBox to run on a background thread

I checked this repo's branches here: https://github.com/alunbestor/Boxer/branches/all , but even the most current has been dormant for 13 months.

Is this work taking place in another repo / branch, where @dreamer and I can stay abreast of? Excited about having more good work flowing into the code-base.

[MaddTheSane]

My fork https://github.com/MaddTheSane/Boxer is where most of the changes can be seen.

[kcgen]

Good stuff @MaddTheSane ! Thanks for the heads-up.

[Megalodon1990]

Sorry i have to ask but i have no idea what i shall do with the git repository. I check it out and now i have MaddTheSane's branch on my system. How can i start/build/compile boxer? Yeah i built some C projects in the past but i am not used to "mac" projects.

[MaddTheSane]

If you have Xcode installed, you should be able to double-click Boxer.xcodeproj and open it in Xcode. Assuming you got all the git subprojects, it should build without any issue.

[WizardOfWor1969]

IMHO It might be nice to have builds on this site, not everyone that wants to use Boxer happens to be a developer. I am not sure how that would work here, but I noticed other projects have a release tab even for pre-release "beta" quality software.

[dreamer]

@MaddTheSane for automatic builds - feel free to copy / lookup examples how to accomplish automatic builds on CI using GitHub Actions from dosbox-staging; our macOS jobs are here. I feel this might be good starting point or initial reference for creating Boxer CI. Also, I feel like maybe this topic should be redirected to an issue in your repo?

[MaddTheSane]

Also, I feel like maybe this topic should be redirected to an issue in your repo?

Agreed.

I'm setting up some CI stuff on my fork at https://github.com/MaddTheSane/Boxer