Apple Silicon support

[stefand]

Neither of us has the necessary hardware yet, and 16kb pages will be a major problem.

[darkbasic]

Neither of us has the necessary hardware yet, and 16kb pages will be a major problem.

You don't have to, using 4K is as simple as:

CONFIG_ARCH_MMAP_RND_BITS_MAX=33
CONFIG_ARCH_MMAP_RND_BITS_MIN=18
CONFIG_ARCH_MMAP_RND_COMPAT_BITS_MAX=17
CONFIG_ARCH_MMAP_RND_COMPAT_BITS_MIN=11
CONFIG_PPC_4K_PAGES=y
# CONFIG_PPC_64K_PAGES is not set
CONFIG_PPC_PAGE_SHIFT=12
CONFIG_FORCE_MAX_ZONEORDER=13
CONFIG_ARCH_MMAP_RND_BITS=18
# CONFIG_VMXNET3 is not set

The previous snipped is for ppc64le, but you can do the same on M1 because 4K is supported natively as well. In fact Rosetta is a 4K application: in MacOS 16K and 4K can coexist.

[stefand]

Yeah but following the Asahi Linux people's statements, while the CPU supports 4k pages, the iommu does not. Linux will have to gain support for per-process page sizes, similar to macos, or will have to deal with a CPU/IOMMU page size mismatch. That's some time away.

[darkbasic]

AFAIK in 2021 they were on their way to fix that issue:

IOMMU 4K patches (in review): The M1 is peculiar in that, although it supports OSes that use either 16K or 4K pages, it really is designed for 16K systems. Its DART IOMMU hardware only supports 16K pages. These chips have 4K support chiefly to make Rosetta work on macOS, but macOS itself always runs with 16K pages – only Rosetta apps end up in 4K mode. Linux can’t really mix page sizes like that and likely never will be able to, so we’re left with a conundrum: running a 16K kernel makes compatibility with older userspace difficult (chiefly Android and x86 emulation), plus distros don’t usually ship 16K kernels; while running a 4K kernel runs into a major mismatch with the DART. This initially seemed like a problem too intractable to solve, but Sven took on the challenge and now has a patch series that makes Linux’s IOMMU support layer play nicely with hardware that has an IOMMU page size larger than the kernel page size! It’s not perfect, as it can’t support a select few corner case drivers (that do things that are fundamentally impossible to support in this situation), but it works well and will support everything we need to make 4K kernels viable.

Is there any news on that?

[stefand]

No real progress on the page size thing: https://arstechnica.com/gadgets/2023/02/linux-is-not-exactly-ready-to-run-on-apple-silicon-but-give-it-time/

[darkbasic]

Apparently things are finally moving:

FEX doesn’t work on standard Asahi Linux kernel builds since we use 16K pages, but 4K page support is not actually that difficult to add… so starting this week, I’m going to be adding 4K support to the Asahi GPU driver and fixing whatever issues I run into along the way, and then we’re going to try running Steam and Proton on it!

https://asahilinux.org/2023/03/road-to-vulkan/

[ToCodeABluejay]

Perhaps if you try a custom kernel build? Will have to give it a try...🤔https://www.reddit.com/r/AsahiLinux/comments/tq575y/possibility_to_install_a_4k_kernel_on_arch/

[ToCodeABluejay]

https://github.com/AsahiLinux/linux/issues/47?fbclid=IwAR2EEOuENLoK9P3QT9gomHIPm31yH1pZrdVXbPb9aH5V5xR8WAQzCQvsvZ4#:~:text=That%20being%20said%2C%20it%20does%20work%20but%20the%20implementation%20is%20a%20bit%20of%20a%20hack%20and%20I%E2%80%99m%20waiting%20for%20another%20series%20by%20Robin%20Murphy%20which%20cleans%20up%20the%20iommu%20API%20before%20I%20continue%20working%20on%20it.

[ToCodeABluejay]

After successfully installing llvm-mingw on Asahi, and building and installing a 4kb kernel, I tried building hangover only to get the following build failure...

   ld.lld: error: undefined symbol: ___wine_import_ucrtbase_name
    >>> referenced by /tmp/libucrtbase-00000000-80cda7.o:(___wine_import_ucrtbase_desc)

    ld.lld: error: undefined symbol: _WinMain@16
    >>> referenced by ../crt/crtexewin.c:67
    >>>               libmingw32.a(lib32_libmingw32_a-crtexewin.o):(_main)
    clang-16: error: linker command failed with exit code 1 (use -v to see invocation)
    winebuild: /usr/bin/i686-w64-mingw32-as failed with status 1
    make: *** [Makefile:157954: dlls/ucrtbase/i386-windows/libucrtbase.a] Error 1
    make: *** Attente des tâches non terminées....

Building with the following release of llvm-mingw-ucrt on Asahi Linux using edge kernel 6.3.0-5 https://github.com/mstorsjo/llvm-mingw/releases/tag/20230603

[owlshrimp]

Apparently things are finally moving:

FEX doesn’t work on standard Asahi Linux kernel builds since we use 16K pages, but 4K page support is not actually that difficult to add… so starting this week, I’m going to be adding 4K support to the Asahi GPU driver and fixing whatever issues I run into along the way, and then we’re going to try running Steam and Proton on it!

https://asahilinux.org/2023/03/road-to-vulkan/

Sadly, this doesn't include any work to fix the IOMMU problem.

I think they looked at reworking core linux code to address the IOMMU, but have not worked to upstream it. Part of that is because the work is extremely invasive and it would be a ton of political work to get linux to accept the patches. But, I think they might also be trying to encourage applications (like chromium, for example) to fix their 4K page assumptions and get better at running on other page sizes.

Sooner or later I guess some kind of solution will need to be found for emulating crusty old 4K x86 code on future computers with larger page sizes. That's probably out of scope for hangover though.

[stefand]

There are hypothetical solutions to the 16kb page size problem.

Windows might use 4kb page sizes, but the API mostly exposes 64k page sizes. VirtualAlloc et al work on 64k boundaries. The reason for this is the historic support for Alpha CPUs on early Windows NT.

The place where you get 4k granularity is mapping PE files and VirtualProtect on existing allocations. Wine's own PE files can be built with 16kb section alignment. I suspect (but don't have any sort of proof) that most applications will be happy by using the union of access permissions on 4 4kb pages that get mapped into one 16kb page (i.e., if 8 kb of read-write .data and 8kb of read-execute .code get mapped into one place, make it a 16kb .rwx page). It might be possible to relocate individual sections if the file contains relocation symbols (although the increase in PIC code means that more and more DLLs don't have reloc symbols because they normally don't need them).

Another option is to to give a merged page minimum permissions (read-only in the above example), catch exceptions and manually handle writes and execute attemps. When an emulator is involved the emulator might be able to help here. This approach has its own problems (e.g. we might have to single-step through code that shares a page with non-executable data), performance may suffer, etc.

And finally, there are copy protection systems that will be very picky about things like this :-(

[hackgrid]

AsahiLina got many steam games running since at least March, so somehow it is possible already :-)

https://www.youtube.com/watch?v=CJSfFzsU75g

[owlshrimp]

AsahiLina got many steam games running since at least March, so somehow it is possible already :-)

https://www.youtube.com/watch?v=CJSfFzsU75g

As far as I know that was on one of the (broken) 4K page kernels.

[hackgrid]

Haha, seemed pretty functional to me :D What was broken? But nevermind, I just mentioned it, because the consensus in this issue was, that this should be not possible, but it seems it is already! :)

[owlshrimp]

Haha, seemed pretty functional to me :D What was broken? But nevermind, I just mentioned it, because the consensus in this issue was, that this should be not possible, but it seems it is already! :)

The IOMMU. Apple's IOMMU hardware only works with 16K pages, and nobody (especially asahi) is willing to ship a kernel with a broken IOMMU.

People have known since the beginning that running a broken 4K kernel would let you run 4K windows emulation. Most of this thread exists because shipping this solution to users is not possible and will not happen.

(The asahi folk have also decided not to upstream their attempt at running the 16K IOMMU with a 4K kernel. It's far too awful to ever be accepted into mainline, and they'd rather see 16K support added to applications.)

[AndreRH]

I thought they work on 4K support for the GPU according to https://asahilinux.org/2023/03/road-to-vulkan/ Anyway, with a 4K support Hangover should run as well as on a 64-Bit Raspberry Pi I assume.

[owlshrimp]

I thought they work on 4K support for the GPU according to https://asahilinux.org/2023/03/road-to-vulkan/

They worked on 4K support for *only* the GPU. The GPU now works on a (broken) 4K kernel, the IOMMU (still) does not.

[hackgrid]

Marcan seems very passionate about gaming on Asahi Linux, so I guess everything will work out, since many things already work or their team is working on that for a long time, so let's just see what the near future brings us :)

[hackgrid]

Just want to mention there is dev work on an interesting solution going on, via a microvm + 4K kernel:

https://bahn.social/@slp@fosstodon.org/111094390741959319 https://twitter.com/slpnix/status/1704612939611718136

[hackgrid]

This is awesome now:

https://sinrega.org/2023-10-06-using-microvms-for-gaming-on-fedora-asahi/

Easy install of Fex/MicroVM stuff with native gpu performance for all! :)