Get a functional ARM64 compiler

[anaisbetts]

Hi, I'm looking into what it would entail to get MSys2 running under ARM64 on Windows, and I wanted to kick off a separate thread with a few questions / making sure I understand the challenges. The biggest one seems to be how to get a functioning compiler working.

From what I understand, at the end of the day, msys2 uses the gcc compiler (via mingw-w64), which doesn't support ARM64 on Windows, so we'd have to build MSys2 with clang. But! The tricky part is, MSys clang package seems to require gcc!

So, it seems that we somehow have to build enough of mingw-w64 using clang cross-compilation to be able to build things with aarch64-w64-windows-gnu, then cross-compile clang targeting the ARM64 mingw-w64. This will eventually be way easier once we've solved the bootstrapping problem and we can just use prebuilt binaries to download ARM64 clang and go.

Once we're able to build ARM64 MSys2 packages, it seems like most of the work will be to make sure they all build under clang instead of gcc, and we'll probably need to also be able to build arm64 binaries from 64-bit Intel since setting up a native ARM64 build machine is probably a non-starter.

Am I missing anything? I'm basically coming into this blind so I may be extremely mistaken / confused! Any guidance / pointers to the right direction would definitely be appreciated 😅

Refs #1276 #1787

[Biswa96]

As discussed in this thread Will There be an ARM/ARM64 Windows Support Soon?, the llvm-mingw can be used in MSYS2 environment to create ARM & ARM64 PE binaries. But it can not be fully replaced with GCC. Most of the packages (e.g. gmp, binutils, coreutils etc.) need some tuning for ARM. But it is possible to invoke llvm-mingw and compile higher level programs.

btw, are you interested to compile any package for ARM?

[anaisbetts]

@Biswa96 My end-goal is to get Git for Windows built for native ARM64, but I'd like to generalize this work / make something maintainable instead of making something really one-off and hacky. I'll have a look at llvm-mingw.

Is ARM64 support something that msys2 maintainers would be willing to host if someone did the initial Work to get things working? I don't mind trying my hand at cleaning up all of the build breaks for clang + llvm, but I can't really set up my own msys2 mirror / infrastructure, etc etc

[lazka]

My end-goal is to get Git for Windows built for native ARM64

@dscho ^ have you looked into ARM64 for git?

[dscho]

There is an active PR to compile Git for Windows for ARM64 using MSVC, and the plan was to drop the executables into an i686 version of Git for Windows (which can apparently be run on ARM64 Windows). That's all we got for now.

[Biswa96]

If git-for-windows and all the dependencies can be compiled using clang then llvm-mingw can also be used. cc. @mstorsjo

[mstorsjo]

There is an active PR to compile Git for Windows for ARM64 using MSVC, and the plan was to drop the executables into an i686 version of Git for Windows (which can apparently be run on ARM64 Windows). That's all we got for now.

That sounds like it should work, yes. Compiling the core git parts with llvm-mingw could also work, as they're already portable if they can be built with MSVC, but whichever toolchain you are most comfortable with for those parts work. The bash/msys underpinnings is the big question, and just shipping i686 binaries for that should be a decent stopgap solution. I've tested 32 bit msys2 on arm64, and iirc it worked just fine, so it didn't do anything that would trip up the emulation.

If git-for-windows and all the dependencies can be compiled using clang then llvm-mingw can also be used.

In general, it's not only about being able to compile things with clang, but also working with lld instead of binutils ld (most high level things work just fine, but there are features of binutils ld that aren't supported in lld when targeting COFF), and a number of other things.

As git-for-windows essentially also contains a copy of msys/msys2 in some form, you'd need support for compiling msys binaries with the llvm-mingw toolchain, and that's not supported yet.

Compiling things for MSYS essentially requires compiling things for the cygwin target, if I've understood things correctly. And clang/llvm does contain some amount of support for targeting cygwin, but it is essentially unmaintained as far as I know - I don't know of anybody using it, and I haven't seen a single patch towards fixing any aspect of it (during the last few years when I've done hundreds of patches to fix MinGW related aspects in LLVM, Clang and LLD).

So far I've had my hands full with just MinGW related bits, so I haven't even started peeking into these things. (Also fwiw, my policy regarding that work so far, has been to work closely with upstream so that llvm-mingw uses a fully unpatched llvm/clang/lld build, and the same goes for mingw-w64. The little bit of customization that there is, is in the scripts that builds and assembles it all, and in tool wrapper scripts.)

Some of the fixes for MinGW directly also benefit Cygwin, so things might have improved implicitly, but other fixes are behind cases like if (mingw) in the codebase. It might be that those also just work if the cases are changed into if (mingwOrCygwin), but there could also be lots of other cases where further changes are needed - and as the general state of cygwin support in llvm is unknown, I haven't spent time along the way to see how my fixes behave in a cygwin context.

As far as I see it, the steps towards MSYS2 (or the msys underpinnings of git-for-windows) on ARM64 would be this:

1. Make sure that clang and lld work for the cygwin target in general (on x86), within a normal cygwin install. For trivial C apps, this might kind of work as is. For large C++ apps, it might require enabling a number of the mingw specific tweaks I've done over the years.
2. Once things works for cygwin (for which there's at least some form of support for in LLVM/Clang), make it work for MSYS2 as well. Not sure if this is a no-op or if it requires some amount of differences for e.g. linking default libraries (with different names).
3. Figure out how to bootstrap the MSYS2 runtime via cross compilation, using the regular GCC used in MSYS2 today. Not sure if this is common knowledge and a commonly done operation, or if MSYS2 only is developed on top of itself. Th
4. Once cross-bootstrapping with GCC works, replace GCC with Clang, LLD and the llvm based runtimes (compiler-rt, libunwind, libcxxabi, libcxx - replacing libgcc and libstdc++). This can probably benefit from cross-wrappers like the ones in llvm-mingw, but with a different target triple than the mingw ones used there. For the runtimes, it can probably also reuse a lot of the build scripts from llvm-mingw. All of this could potentially also be merged into llvm-mingw in the end.
5. In the cross-setup, target aarch64-pc-msys instead of x86_64-pc-msys. This can potentially be the step with the least amount of work - unless there are cases in the msys runtime with assembly or other architecture specific bits. I'd imagine there can be very architecture specific bits in e.g. the fork emulation in the base of the runtime - in that case this step can potentially be hard.

So as you can see, for the general "MSYS2 on ARM64", there's a lot to do and figure out, before questions like hosting and infrastructure are relevant.

For just compiling normal mingw based projects for ARM64, llvm-mingw works fine as is right now, both for compiling on MSYS2 on x86, or for cross compiling from a linux environment.

[harrisonmetz]

This question is probably naive (and an educationally focused question), but why can't llvm nor gcc be built with msvc? Is it the fact that the compiler code itself can't be compiled, or that the issue is just lack of target support for windows arm64? Thanks.

[mstorsjo]

This question is probably naive (and an educationally focused question), but why can't llvm nor gcc be built with msvc?

Llvm can be built with msvc. Gcc probably can't (I don't think it's supported or commonly done), although it might be possible with some effort.

Is it the fact that the compiler code itself can't be compiled, or that the issue is just lack of target support for windows arm64? Thanks.

Gcc doesn't support generating code for windows/arm64, so even if you'd use a different compiler for having a gcc running on windows/arm64, you could only have a gcc producing code for a different OS or architecture.

Llvm supports generating code for windows/arm64, and can itself also run on windows/arm64, if compiled with either msvc or clang/llvm. https://github.com/mstorsjo/llvm-mingw provides this; both cross compilers for running on other OSes producing code for windows/arm64, and for running directly on windows/arm64 as well.

The discussion above is about how to get the msys2 runtime built with clang/llvm.

[harrisonmetz]

Ok, thanks for the explanation.

[Biswa96]

One question, if someone has knowledge can Windows 10 ARM64 support be added in whole GCC toolchain without relying on clang? Or is it impossible to do so?

[mstorsjo]

One question, if someone has knowledge can Windows 10 ARM64 support be added in whole GCC toolchain without relying on clang? Or is it impossible to do so?

Of course it's possible, but you should not think that it's trivial - it's not; I added support for it in Clang/LLVM.

Before Windows on ARM64, there were also efforts for Windows on ARM32 (which Clang also supported before), and a GCC maintainer was involved and wrote some initial proof of concept patches for GCC to target Windows on ARM32, but those patches were never completed or upstreamed. So for someone with less experience of GCC it's certainly an even bigger task.

[Myriachan]

Even having an GNU assembler for ARM64 that supports Windows ARM64 .pdata is hard enough...

[stackTom]

For just compiling normal mingw based projects for ARM64, llvm-mingw works fine as is right now, both for compiling on MSYS2 on x86, or for cross compiling from a linux environment.

So would it be feasible to compile GMP, MPFR, and MPC for arm64? I got static and dynamic x86 and x64 bit builds of these libraries with msys2, and was wondering how feasible ARM would be.

[stackTom]

For just compiling normal mingw based projects for ARM64, llvm-mingw works fine as is right now, both for compiling on MSYS2 on x86, or for cross compiling from a linux environment.

So would it be feasible to compile GMP, MPFR, and MPC for arm64? I got static and dynamic x86 and x64 bit builds of these libraries with msys2, and was wondering how feasible ARM would be.

It is possible, see: https://github.com/mstorsjo/llvm-mingw/issues/131

I don't have a windows for arm machine to test if the built libraries run, but they do successfully compile and can be linked in visual studio.

[Biswa96]

If Git in Windows can be compiled without msys2/cygwin then I can try to build it for Windows ARM64. In other words, if mingw-w64-git package can be created then it is possible to port it to ARM with no or little effort using llvm-mingw toolchain.

[dscho]

Git for Windows can indeed be compiled without MSYS2/Cygwin, at least with the upcoming CMake support. Without CMake support, it would require a GNU make, I think.

[driver1998]

Git for Windows can indeed be compiled without MSYS2/Cygwin, at least with the upcoming CMake support. Without CMake support, it would require a GNU make, I think.

GNU make is fine, we can emulate make, or build the thing in WSL (which is native)

[dscho]

@driver1998 feel free to test this hypothesis... :grinning:

[dennisameling]

Just leaving this here for reference: we're currently working on a native Git for Windows executable (git.exe etc.) while relying on mingw32 for the packages (bash/curl/etc.). This could be a good first step prior to having an actual compiler for arm64, and already leads to a ~2x speed improvement for basic Git commands 🚀

Updates here: https://github.com/git-for-windows/git/issues/2346#issuecomment-751687623

[jeremyd2019]

1. Make sure that clang and lld work for the cygwin target in general (on x86), within a normal cygwin install. For trivial C apps, this might kind of work as is. For large C++ apps, it might require enabling a number of the mingw specific tweaks I've done over the years.
2. Once things works for cygwin (for which there's at least some form of support for in LLVM/Clang), make it work for MSYS2 as well. Not sure if this is a no-op or if it requires some amount of differences for e.g. linking default libraries (with different names).

There is now a clang/lld packaged on MSYS2, thanks to @mati865 🚀

[zap8600]

Hello! Has anyone built any the packages below for WoA?

make bison flex gawk gmp mpc mpfr texinfo

I'm building a OS and using my Raspberry Pi 4 as my work PC (because it is the only decent PC in my house). I want to recompile the compiler for Aarch64 made by @mati865 so that it'll be armv8, Cortex a72, and stuff like that. Do I need special files for compiling a new compiler? What compiler flags do I use?

[stackTom]

Hello! Has anyone built any the packages below for WoA?

make bison flex gawk gmp mpc mpfr texinfo

I'm building a OS and using my Raspberry Pi 4 as my work PC (because it is the only decent PC in my house). I want to recompile the compiler for Aarch64 made by @mati865 so that it'll be armv8, Cortex a72, and stuff like that. Do I need special files for compiling a new compiler? What compiler flags do I use?

I've built gmp, mpc, mpfr for arm64. Never tested on windows on arm though, but I assume it'll work.

[zap8600]

Thanks!

From, Zane Post

On Fri, Dec 17, 2021, 20:27 stackTom @.***> wrote:

Hello! Has anyone built any the packages below for WoA?

make bison flex gawk gmp mpc mpfr texinfo

I'm building a OS and using my Raspberry Pi 4 as my work PC (because it is the only decent PC in my house). I want to recompile the compiler for Aarch64 made by @mati865 https://github.com/mati865 so that it'll be armv8, Cortex a72, and stuff like that. Do I need special files for compiling a new compiler? What compiler flags do I use?

I've built gmp, mpc, mpfr for arm64. Never tested on windows on arm though, but I assume it'll work.

— Reply to this email directly, view it on GitHub https://github.com/msys2/MSYS2-packages/issues/1888#issuecomment-997127667, or unsubscribe https://github.com/notifications/unsubscribe-auth/AWYQOAUAVFVTVUXSX6OWZV3URPWQLANCNFSM4LEAVP4A . You are receiving this because you commented.Message ID: @.***>

[zap8600]

... I forgot... this isn't github.

From, Zane Post

On Fri, Dec 17, 2021, 20:28 Zane Asher Post @.***> wrote:

Thanks!

From, Zane Post

On Fri, Dec 17, 2021, 20:27 stackTom @.***> wrote:

Hello! Has anyone built any the packages below for WoA?

make bison flex gawk gmp mpc mpfr texinfo

I'm building a OS and using my Raspberry Pi 4 as my work PC (because it is the only decent PC in my house). I want to recompile the compiler for Aarch64 made by @mati865 https://github.com/mati865 so that it'll be armv8, Cortex a72, and stuff like that. Do I need special files for compiling a new compiler? What compiler flags do I use?

I've built gmp, mpc, mpfr for arm64. Never tested on windows on arm though, but I assume it'll work.

— Reply to this email directly, view it on GitHub https://github.com/msys2/MSYS2-packages/issues/1888#issuecomment-997127667, or unsubscribe https://github.com/notifications/unsubscribe-auth/AWYQOAUAVFVTVUXSX6OWZV3URPWQLANCNFSM4LEAVP4A . You are receiving this because you commented.Message ID: @.***>

[jeremyd2019]

You mean like https://packages.msys2.org/package/?repo=clangarm64 ? If you mean actual msys2, the cygwin runtime is not compatible with ARM, and msys2-runtime is a fork of cygwin, so that's probably not happening any time soon. On Windows 11, x64 msys works reasonably well under emulation.

[dscho]

the cygwin runtime is not compatible with ARM

And this mail on Dec 8, 2021 from the Cygwin lead suggests that probably won't change any time soon (and they'd require mingw-w64 to support ARM64 first).

[ZachBacon]

the cygwin runtime is not compatible with ARM

And this mail on Dec 8, 2021 from the Cygwin lead suggests that probably won't change any time soon (and they'd require mingw-w64 to support ARM64 first).

I mean, technically it does support it. More so using the clang compiler. But that's still relatively good news since the arm side of mingw has been getting better supported lately.

[mstorsjo]

the cygwin runtime is not compatible with ARM

And this mail on Dec 8, 2021 from the Cygwin lead suggests that probably won't change any time soon (and they'd require mingw-w64 to support ARM64 first).

Just for clarity; mingw-w64 supports ARM64 just fine - I wouldn't consider it experimental. Since maybe 3 years or so, it's on par with the support for x86. It has of course much less track record than the x86 parts, but I'm not aware of any outstanding issues.

As potential follow-up to that post though (I'm not subscribed to that list so I can't easily reply):

• Toolchains (binutils, gcc, gdb) targeting aarch64-pc-cygwin and aarch64-w64-mingw32.

  I don't know if the mingw toolchain already exists, but if so, the Cygwin toolchain could be derived from there. The differences would be basically the same as on x86_64 (sizeof(long), different default link libs, etc)

This is indeed the big blocker, as GCC/binutils doesn't support it. Clang/LLVM does support windows on aarch64 just fine, but the cygwin target (in general even for x86) in Clang/LLVM is essentially unmaintained. So first someone would need to pick that up, and try cross-building a working x86 cygwin environment using Clang/LLVM, and fix up whatever cases need fixing (not sure if that target in LLVM has worked at any point or not, but it has most probably bitrotted a little). Once that works, one could try targeting aarch64 instead (and fill with all the arch specific assembly bits in cygwin).

[mati865]

This is indeed the big blocker, as GCC/binutils doesn't support it. Clang/LLVM does support windows on aarch64 just fine, but the cygwin target (in general even for x86) in Clang/LLVM is essentially unmaintained. So first someone would need to pick that up, and try cross-building a working x86 cygwin environment using Clang/LLVM, and fix up whatever cases need fixing (not sure if that target in LLVM has worked at any point or not, but it has most probably bitrotted a little). Once that works, one could try targeting aarch64 instead (and fill with all the arch specific assembly bits in cygwin).

Well, we have half broken Cygwin like LLVM toolchain: https://github.com/msys2/MSYS2-packages/pull/2230 I didn't bother with anything more than needed for bootstrapping LLVM for CLANG* environments. IIRC C++ stdlib via libstdc++ is totally broken.

[zap8600]

Hello! The C Compiler @mati865 made works really well! I compiled libsdl with it. Libsdl works perfectly, so the compiler worked. There is one thing though. When I configured libsdl the first time, it said the host was unsupported. When I added "--host=aarch64-w64-mingw32" and "--target=aarch64-w64-mingw32" it worked.

[zap8600]

Hi, it is me. Does anyone know if there are cmake binaries for WoA64?

[jeremyd2019]

Does anyone know if there are cmake binaries for WoA64?

https://packages.msys2.org/package/mingw-w64-clang-aarch64-cmake?repo=clangarm64

[mstorsjo]

Hi, it is me. Does anyone know if there are cmake binaries for WoA64?

I made a build last year, https://martin.st/temp/cmake-mingw-aarch64.zip. IIRC it should build fine without further patches after https://gitlab.kitware.com/cmake/cmake/-/commit/1aba3a8367f8e9a6f12e409f170008d930f87855.

[zap8600]

Hi. Thanks for the cmake! However, I have a new compiler problem. The OpenGL library file doesn't exist. The header files are there, just not the library. Also, does anybody know how I can get a copy of OpenGLES 2? I want to use the hardware acceleration that the Raspberry Pi 4 has, but I can't find a way to get the library or to even get the source code.

On Tue, Jan 4, 2022 at 3:46 PM Martin Storsjö @.***> wrote:

Hi, it is me. Does anyone know if there are cmake binaries for WoA64?

I made a build last year, https://martin.st/temp/cmake-mingw-aarch64.zip. IIRC it should build fine without further patches after https://gitlab.kitware.com/cmake/cmake/-/commit/1aba3a8367f8e9a6f12e409f170008d930f87855 .

— Reply to this email directly, view it on GitHub https://github.com/msys2/MSYS2-packages/issues/1888#issuecomment-1005194276, or unsubscribe https://github.com/notifications/unsubscribe-auth/AWYQOAVGJH6UB3O3P3ADK6DUUNTE7ANCNFSM4LEAVP4A . You are receiving this because you commented.Message ID: @.***>

[jeremyd2019]

See https://github.com/mstorsjo/llvm-mingw/issues/209 (the latest mingw-w64-clang-aarch64-crt-git has libopengl32.a), but I wouldn't bet on 3d acceleration drivers for Windows on Raspberry Pi.

[zap8600]

Hey @stackTom what header and library files do I need for mpc, mpfr, and gmp? When I compile a library, it goes to /usr in MSYS i686, so I have to copy the files. Or do you have already compiled libraries I can use?

On Tue, Jan 4, 2022 at 8:02 PM jeremyd2019 @.***> wrote:

See mstorsjo/llvm-mingw#209 https://github.com/mstorsjo/llvm-mingw/issues/209, but I wouldn't bet on 3d acceleration drivers for Windows on Raspberry Pi.

— Reply to this email directly, view it on GitHub https://github.com/msys2/MSYS2-packages/issues/1888#issuecomment-1005317888, or unsubscribe https://github.com/notifications/unsubscribe-auth/AWYQOAXRYJYG6TSZJLI4NCLUUORFDANCNFSM4LEAVP4A . You are receiving this because you commented.Message ID: @.***>

[NiQ1]

Hi. Thanks for the cmake! However, I have a new compiler problem. The OpenGL library file doesn't exist. The header files are there, just not the library. Also, does anybody know how I can get a copy of OpenGLES 2? I want to use the hardware acceleration that the Raspberry Pi 4 has, but I can't find a way to get the library or to even get the source code. … On Tue, Jan 4, 2022 at 3:46 PM Martin Storsjö @.> wrote: Hi, it is me. Does anyone know if there are cmake binaries for WoA64? I made a build last year, https://martin.st/temp/cmake-mingw-aarch64.zip. IIRC it should build fine without further patches after https://gitlab.kitware.com/cmake/cmake/-/commit/1aba3a8367f8e9a6f12e409f170008d930f87855 . — Reply to this email directly, view it on GitHub <#1888 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AWYQOAVGJH6UB3O3P3ADK6DUUNTE7ANCNFSM4LEAVP4A . You are receiving this because you commented.Message ID: @.>

Windows on ARM does not natively support OpenGL and the x86 opengl.dll (in Windows\SysWOW64) is actually an OpenGL API wrapper around DX12 if I understand correctly.

Microsoft has released the OpenCL™ and OpenGL® Compatibility Pack (can be installed for free from the Windows store), which should have ARM OpenGL libraries but mind that those are also wrappers around DX12.

[jredfox]

how is this not fixed and why has it taken 3 years? I am simply looking for a mingw compiler Host:x64 Target:arm64 win32 api and thread model:msvcrt which doesn't seem to exist from either you guys or mingw. Windows arm has been out since 8.1 at least why hasn't there been a compiler for it yet? The surface is pretty popular and has been out for just about as long

[ZachBacon]

how is this not fixed and why has it taken 3 years? I am simply looking for a mingw compiler Host:x64 Target:arm64 win32 api and thread model:msvcrt which doesn't seem to exist from either you guys or mingw. Windows arm has been out since 8.1 at least why hasn't there been a compiler for it yet? The surface is pretty popular and has been out for just about as long

Excuse you, but there is a functional arm64 compiler in the form of the clangarm64 subsystem, though mind you the compiler itself is native arm64 so you need an arm64 host to run it. But it's entirely there.

[jredfox]

@ZachBacon clang cross platform arm64 is not arm64 windows compiler. it uses a POSIX thread model not windows(UWP etc...) or win32 thread model. It's also not optimized for windows. Meaning it could never work under WinPE nor be great in performance then if it just supported windows formats.

that being said I should have just posted this under MingW issues as you guys are pretty confusing compared to MingW. I use eclipse which simply works when compiling unlike 2022 visual studio

The only thing that worked for me was Visual Studio 2017 downgrade --> arm64 and it only works compiling it for the release and then changing multi threading to /MT

[MehdiChinoune]

@jredfox the compiler you are asking for is already available under MSYS2 clangarm64 environment. That what @ZachBacon Was trying to tell you.

[jredfox]

@jredfox the compiler you are asking for is already available under MSYS2 clangarm64 environment. That what @ZachBacon Was trying to tell you.

Read my comment clang arm64 is not windows arm64. It uses POSIX thread model will not work for winpe

[mati865]

clangarm64 uses Win32 threading model, not POSIX/winpthreads one.

[jeremyd2019]

I am simply looking for a mingw compiler Host:x64 Target:arm64 win32 api and thread model:msvcrt which doesn't seem to exist from either you guys or mingw.

This project doesn't really supply cross compilers at this point. I've done some playing with this in msys2/MINGW-packages#8762 but it just doesn't really fit in well with how packages are structured here (see also https://github.com/msys2-arm/cross-clang/releases/tag/repo you'd want to install mingw-w64-clang-x86_64-cross-clang-aarch64). Perhaps what you really want is llvm-mingw?

[jeremyd2019]

It uses POSIX thread model will not work for winpe

Wait, does winpthread not work under WinPE? That seems odd.