Steps for bringing the newest build to OSX

[aktau]

Hey guys, this is just some exploratory thinking I'm doing but:

Seeing as PCSX2 supports both Linux and Windows and is (as far as I can see) based mostly on cross-platform libs. This leads to the suspicion that a proper port to OSX wouldn't be a herculanean task. The (free) OSX 10.9 supports OpenGL 4.1 and the drivers are "ok" (not great). Given the fact that 10.9 is free and the OpenGL implementation is far less buggy than earlier incarnations, I see no reason to support anything earlier.

Instead of plowing through the codebase, I'd like to ask the devs: what do you see as the major hurdles that someone porting PCSX2 would encounter?

EDIT: I'm aware of earlier versions of PCSX2 being made available for OSX and I've noticed some vestigial bits in the CMake files et al.

[gregory38]

Porting gsdx ogl to osx won't be difficult. It mostly need the bits to create the ogl surface and context. Then various ifdef linux must be checked. It would also need pthread (or a port to std::thread).

[aktau]

Yes indeed, it's usually the surface creation, small driver bugs and different GL context <-> threading interactions that cause the problems on the 3D side. In my own projects I tend to use SDL2 to get a surface. This works very well over linux/osx/win without code changes on my part.

I've noticed in build.sh that there is actually a --sdl2 option already. Since I'm very unfamiliar with wxWidgets ('m a C dev at heart), does wxWidgets use SDL2 behind the scenes for cross-platform GL surface creation? A bit of googling says that it doesn't, but I could be wrong. Is the --sdl2 option actually in use?

[aktau]

And after trying to build it I suddenly realize why this might be harder than anticipated: pcsx2 is 32-bit only (CMake tells me). That usually causes no end of troubles on systems where 32-bit is uncommon (such as OSX 10.9).

[gigaherz]

Porting to X64 would have exactly the same issue as porting to ARM: most of the core emulation (cpu, memory, VUs) is only designed for x86 and would have to be rewritten. There may be LESS work involved in porting to x64, but it would still be a huge project.

[aktau]

@gigaherz yes I've read up on it on the forums. I wasn't advocating porting to 64-bit despite possible (dis)advantages, as it would be a lot of work as you said. Just mentioning to myself and others who would want to attempt the same that this might be one of the reasons why pcsx2 doesn't have an up-to-date OSX build.

[gigaherz]

Same here, I was just mentioning, in case anyone wondered, that the porting would require a huge effort. I guess I should have specified ;P

[gregory38]

The creation of surface/context is nothing. You get the glx file and you replace glx with the name of osx api. 90% jobs done. It took me 1 or 2 days to port linux ogl to windows (and I'm not even sure I booted on a win machine to test the code). Besides you can based the code on zzogl mac. Note: gsdx normally support egl too but I haven't test it for a long time. Note2: yes sdl2 can do some stuff. But we don't want an extra dependency on windows. Besides it creates subtle issues for packaging.

[aktau]

The creation of surface/context is nothing

After looking at the situation a bit more, I agree. By the way: as far as I know EGL on apple is a bit crippled, better to use something like sdl2 or if you prefer the cocoa api. I usually prefer sdl2 because I like to avoid working with Objective C whenever I can. What do you mean by extra dependency? If people don't want to, sdl2 doesn't need to be included for windows. Not sure what packaging issues you guys face, but in my projects I just statically link with sdl2, no problems afterwards.

All that seems to be just peanuts in comparison to the 64-bit issue though, that's the real killer because it touches the most complex part of the code.

[gregory38]

Wx just use std os api as sdl. However wx could potentially depend on sdl. That mean you also need to package wx. Anyway it isn't the linux way.

By the way when I said surface, I meant the gl back buffer not the window. The window is already created by wx. So it only miss CGL (hum maybe C stand for cocas :p). Either you copy the one from zzogl and you rename a couple of variable. Or you could use sdl on mac.

Don't you have fat binary on osx which support both x86 and x86-64.

Note: egl is the future (even if no drivers support it yet). Gsdx only does the minimum with egl, it would work with a primitive implementation (which is likely a copy/paste of glx/wgl/cgl).

[aktau]

Don't you have fat binary on osx which support both x86 and x86-64.

Possibly, although in the age of OSX 10.9 that's not that relevant anymore, almost anyone will have x86-64 chips and installs. It wouldn't matter anyway. The true problem is the dependencies, you can't freely mix/link 32-bit and 64-bit libraries. Since modern OSX is very 64-bit focused, you don't generally have something like multi-arch. Moreover, OSX 10.9 does not have a 32-bit variant. So you can't use the same trick as on linux:

1. Build on a 32-bit linux system
2. Run on a 64-bit linux system with multi-arch

This doesn't work for modern OSX. So the options for a native build are:

• Build on older, 32-bit OSX versions. Or...
• Make sure everything builds as 64-bit. Or...
• Split out the 32-bit specific parts (recompiler et al.) and the 64-bit parts and make them communicate via IPC (easy to do with the likes of nanomsg/libuv/..., works cross-platform). Or..
• ... (I've probably missed something)

I'm not really sure what would be most feasible between the last two.

[gregory38]

What is the issue compiling 32bits code? running 32bits code? both?

[aktau]

What is the issue compiling 32bits code? running 32bits code? both?

Running 32-bits code seems to be more or less ok, at least for console programs. I'm not sure how GUI programs deal with it, I don't know if apple still deploys 32-bit .dylibs/frameworks for cocoa and the likes. For a simple console program one just needs to supply the -m32 flag to both the linker and the compiler. (EDIT: I just tested with a very simple AppKit binary, and it still seems to work with the -m32 flag, which probably means that apple distributes some important .dylibs as universal binaries)

It gets a bit nastier when you're using lots of libraries though. This basically means that for a 32-bit OSX build, all libraries would have to be compiled on the spot by the buildsystem instead of downloaded with a package manager (like the populer homebrew). So there would have to be a shellscript/makefile/cmake-file that downloads and compiles the dependencies with the right flags. A bit like neovim does when it can't find the right dependencies (libuv and luajit).

[chrisballinger]

As far as I know you can still compile code for i386 in clang with -arch i386. The system shared libraries (e.g. those in /usr/lib) are available as fat binaries, even on 10.9. You'd just need to explicitly compile any 3rd-party dependencies for i386 and it would work just fine.

[chrisballinger]

10.9 only supports OpenGL 4.1, with some 4.2 extensions.

• https://developer.apple.com/graphicsimaging/opengl/capabilities/index.html
• http://www.openglsuperbible.com/2013/11/11/porting-samples-to-mac/

[aktau]

You'd just need to explicitly compile any 3rd-party dependencies for i386 and it would work just fine.

Exactly, compiling every non-system dependency is definitely a prerequisite. In my very simplistic tests with some Cocoa things I only needed to specify -m32, how is -arch i386 different?

10.9 only supports OpenGL 4.1, with some 4.2 extensions.

Indeed, is that not sufficient for PCSX2? What kind of GL version do gdsx-gl or zzogl expect?

[gregory38]

Gl must be fine. 3.3 is mostly enough. I used various gl4 functions but they aren't yet mandatory. Actually I managed to run gsdx on gles3 on the past. Now it is bitrotten&broken.

Extension checking is done on GLLoader.cpp. If you can generate me a list of extension supported I can tell you the limitation.

[aktau]

Extension checking is done on GLLoader.cpp. If you can generate me a list of extension supported I can tell you the limitation.

Is the list provided by @chrisballinger not sufficient? Should I check on my Mavericks unit? By the way, you should not take the HD Graphics 3000/4000 into account as they are the integrated cards that are not used for 3D apps. (this can be forced by the user, but is not the default).

[gregory38]

Presentation of the list suck. They could have sorted it by gl version. Anyway it would be fine, it will use several slower code path but nothing critical.

However I do hope they release a newer version that support a better glsl syntax. So I can drop various useless code.

Note for debugging: they are option to autodetect/enable/disable extension (-1/0/1) on gsdx.ini. Just disable geometry shader and enable shader debug too. Note2: convert shader will fail to compile just comment the bad code.

[aktau]

However I do hope they release a newer version that support a better glsl syntax. So I can drop various useless code.

Could you detail what that is please? (I'm curious). As I understand you now, #version 410 is not enough? Most "modern" glsl features do seem to work in my tests.

[gregory38]

Version 330 is enough + some extensions. See that http://www.opengl.org/registry/specs/ARB/shading_language_420pack.txt

For example it allow to set an index of various parameter. Current method is to let the compiler set a random position. Then get current location of the parameter program, and then set it. Lots of work for nothings.

Those extensions are only useful to make the code faster and cleaner. It would work flawlessly without them.

Edit: just add the window creation with SDL2, and it will work (minus geometry shader and a shader that use 420_pack feature).

[aktau]

For example it allow to set an index of various parameter. Current method is to let the compiler set a random position. Then get current location of the parameter program, and then set it. Lots of work for nothings.

Yes, for uniforms that was already possible on OSX 10.7

#version 150
#extension GL_ARB_explicit_attrib_location : enable

However it doesn't seem to be possible with uniform buffers yet, you would need OpenGL 4.2. As far as I can see, OSX 10.9 doesn't support that extension.

[gregory38]

attrib is attribut not uniform. Single uniform need 4.3.

[aktau]

attrib is attribut not uniform. Single uniform need 4.3.

You're right, I had misread, explicit uniforms are indeed an OpenGL 4.3 feature. That said, these things aren't the most important missing things, they can be fixed with some boiler-plate on the client-side and by using fixed names in the shaders (which is a good idea anyway). I'm more "worried" about features in 4.2+ that can't easily be emulated or that provide a nice speed boost.

[gregory38]

Gl must be fine. 3.3 is mostly enough. I used various gl4 functions but they aren't yet mandatory. Actually I managed to run gsdx on gles3 on the past. Now it is bitrotten&broken.

And

Those extensions are only useful to make the code faster and cleaner. It would work flawlessly without them.

I clearly said that it would work.

[aktau]

You did indeed, I should've said "was worried" instead of "am worried".

[aktau]

Btw, you're probably already aware but this GDC presentation about fast, modern OpenGL by some driver authors just came online for free: http://gdcvault.com/play/1020791/. I also found it really cool to look at Dolphin's source code to see how they handle running on different levels of OpenGL: https://code.google.com/p/dolphin-emu/source/browse/Source/Core/VideoBackends/OGL/StreamBuffer.cpp

[morenoh149]

bumping this. Sounds complicated but if the work is outlined I wouldn't mind taking a shot at it.

[aktau]

@morenoh149 from a high-level perspective here's what I can see:

• making sure everything that the cmake buildsystem builds has the flag -m32 passed to it.
• downloading some external dependencies like wxWidgets/sdl2 and compiling them as 32-bits as well.
• selecting a minimal set of plugins that should work on OSX without many changes, which can be used as a bootstrap base. I'm sure @gregory38 has some good insights here. (building the full set of plugins is going to be madness at first anyway, let's not attempt that).

[gregory38]

Well use null plugins :) Onepad is based on SDL so it could work. GSdx can be run in standalone mode with one major difference. The replayer uses GSopen1 (window created by GSdx) whereas the standard behavior is GSopen2 (window created by PCSX2)

See plugins/GSdx/linux_replay.cpp. Then you can use this kind of command

pcsx2_GSReplayLoader libGSdx-0.1.16.so   ./trace_me/colin_bencg_25fps.gs  gsdx_ini_dir/

You create gs files on windows/linux with ctrl/shift/f8, they will be written in the snapshot directory

[uyjulian]

you may have to edit the cmake files to make it work, change the linux to mac parts, remove the gtk2 requirement (wx will use cocca), and don't link with "libc" (librt is missing) there's a lot more to do to make the cmake files compatible with macs.

[uyjulian]

Oh, also, the asm file (.S) do not compile on newer versions of OSX, 'cause clang clang clang.

[aktau]

Oh, also, the asm file (.S) do not compile on newer versions of OSX, 'cause clang clang clang.

That might be temporarily worked around by just requiring gcc. Real easy with homebrew.

[uyjulian]

ASM files: https://github.com/PCSX2/pcsx2/blob/master/pcsx2/x86/aVUzerorec.S https://github.com/PCSX2/pcsx2/blob/master/pcsx2/x86/ix86-32/aVif_proc-32.asm https://github.com/PCSX2/pcsx2/blob/master/pcsx2/IPU/yuv2rgb.asm also zzogl/zerogs has asm too. The rest of the asm: https://github.com/PCSX2/pcsx2/search?p=1&q=__asm__&type=Code (inline asm) Also, macs/FreeBSD have some platform specific asm, asm may have to be adjusted a bit.

[uyjulian]

On the topic of assemblers... gcc will still use the system assembler. Apple Inc version cctools-855, GNU assembler version 1.38 and it will give the same errors as clang.

[aktau]

I used to have a script at /usr/local/bin/as which used to refer to the GNU as (gas) because the old-ass apple assembler didn't support AVX instructions. So I know gas is somewhere...

[uyjulian]

I don't think GNU as generates code required for Macs (Mach-O) I'm not sure if anything has changed, however.

[aktau]

I'm reasonably sure it does, as I've used it for my binaries up until OSX 10.9 (where the native as was upgraded).

[uyjulian]

Ah, later versions of binutils support Mach-O (i.e. 2.2)

[uyjulian]

However, I don't know how to get binutils to build GNU as on mac.

[uyjulian]

Found out how to make gas build, needs to be built for 32bit.

https://gist.github.com/uyjulian/c7c759127d303d4b4499

Point homebrew to that file. Ex: brew install https://gist.githubusercontent.com/uyjulian/c7c759127d303d4b4499/raw/39c0f2d6867d99db3f1a2a1ee3ce5965e6de6290/binutils-gas.rb Figured that out by reading configure.ac in binutils.

[uyjulian]

I would think rewriting the asm files for mac will be easier than fiddling for mac versions of binutils/gcc. GCC won't see that the version of gas is the binutils version, not the cctools version. Or, throwing out all this 32-bit lib madness would make things wayyy easier, just use IPC/shared memory.

[chrisballinger]

Would this be of any help? https://github.com/libav/gas-preprocessor

[uyjulian]

Yeah, that will be useful.

Once I get my 16GB of RAM, I can start working on PCSX2 (because I have enough memory to run a Linux VM).

[gregory38]

On the earlier asm list most of them are useless. Fps2bios -> not used SuperVu -> deprecated On others be sure they are really used, sometimes it just dead code.

[uyjulian]

Guess I can clean up those parts.

[drcreek]

Has there been any progress with this? I'm just wondering if it would be possible to try and build it myself just to mess around with.

[uyjulian]

There's libaio and librt, which aren't available on Macs, so we need replacements for those.

[drcreek]

Does anything in Yosemite help fill in the gaps with this? Like the improved OpenGL support. Also, I have very little experience but I think Transmission for OSX uses something else as a libaio replacement. But I'm not sure exactly what they use, I'm a web developer so the details of this stuff is a little outside my area.

[aktau]

As far as I know, there are no OpenGL advancements on Yosemite (it's still 4.1), and there's not even any more extensions. Perhaps the drivers are more stable?

As for libaio, I'm not sure how PCSX2 is using it, but at the neovim project we're using libuv on win/linux/mac and it works great.

[juhalaukkanen]

I have been working on porting code to OSX. Here's current status.

DONE: -cmake build using wxgtk-2.8 from macports -PCSX2 Core with interpreter core and null SPU/GS plugins.

TODO: -Async IO port for AsyncFileReader -GSdx to use Apple's GL interface GL/gl.h (OSX X11 GL not feasible, only OpenGL 2.1 ctx is supported)

BUGS: -memcpy optimizations -SPU2x linker issue -nasty side effects related to using dispatch/dispatch.h for semaphores -Recompiler doesn't work which seems to be a mmap issue..

FINAL TASKS: -cmake bake so that output is "PCSX2.app" with all dependencies

I will post a pull req once I feel this is in a more mature state.