Initial Windows on ARM64 SDK Support

hmelder commented 11 months ago

This pull request adds initial support for the Windows on ARM platform.

The following build dependencies are required:

MSVC ARM64 > 14.33 as it appears to not have armasm64.exe required by libffi
A native ARM64 CMake installation

My Visual Studio Configuration for reference (17.9.0 Preview 1.1):

There are two pending pull requests, which fix some build system configuration issues:

https://github.com/glennrp/libpng/pull/503 (Edit 20231207: Maintainer made more general fixes. Now in Upstream!)
https://github.com/apple/swift-corelibs-libdispatch/pull/803

Merged Pull Requests:

https://github.com/gnustep/libobjc2/pull/249

I have included the libdispatch patch, but it would be better to wait until the PR is merged.

hmelder commented 11 months ago

Here the results from a GNUstep Base test run:

   8299 Passed tests
     65 Dashed hopes
      8 Skipped sets
      4 Failed tests
      3 Failed files

Failed Tests:

NSURL/test00.m
NSURLConnection/test03.m
NSURLConnection/test04.m
NSURLSession/test02.m
NSURLSession/test03.m

GNUstep Base Test Run Logs

```sh # gnustep-tests base You forgot to set your GNUSTEP_MAKEFILES environment variable. Setting it to /c/GNUstep/arm64/Debug/share/GNUstep/Makefiles during this test run. --- Running tests in base/coding --- --- Running tests in base/Functions --- --- Running tests in base/GarbageCollection --- --- Running tests in base/GSMime --- --- Running tests in base/GSTLS --- base/GSTLS/basic.m: Skipped set: basic.m 45 ... TLS support disabled --- Running tests in base/GSXML --- --- Running tests in base/headers --- --- Running tests in base/KVC --- --- Running tests in base/NSAffineTransform --- --- Running tests in base/NSArchiver --- --- Running tests in base/NSArray --- --- Running tests in base/NSAttributedString --- --- Running tests in base/NSAutoreleasePool --- --- Running tests in base/NSBundle --- base/NSBundle/resources2.m: Skipped set: resources2.m 107 ... Adding a run-time search path is not supported on Windows. --- Running tests in base/NSCache --- --- Running tests in base/NSCalendar --- --- Running tests in base/NSCalendarDate --- --- Running tests in base/NSCharacterSet --- --- Running tests in base/NSConnection --- --- Running tests in base/NSCountedSet --- --- Running tests in base/NSData --- base/NSData/additions.m: Skipped set: additions.m 70 ... zlib support disabled --- Running tests in base/NSDate --- --- Running tests in base/NSDateFormatter --- --- Running tests in base/NSDictionary --- --- Running tests in base/NSDistributedLock --- --- Running tests in base/NSException --- --- Running tests in base/NSFastEnumeration --- --- Running tests in base/NSFileHandle --- --- Running tests in base/NSFileManager --- --- Running tests in base/NSHashTable --- --- Running tests in base/NSHost --- --- Running tests in base/NSHTTPCookie --- --- Running tests in base/NSIndexPath --- --- Running tests in base/NSInvocation --- --- Running tests in base/NSInvocationOperation --- --- Running tests in base/NSJSONSerialization --- --- Running tests in base/NSKeyedArchiver --- --- Running tests in base/NSLocale --- --- Running tests in base/NSLock --- base/NSLock/RecursiveLock.m: Skipped set: RecursiveLock.m 36 ... mutex ownership feature not available --- Running tests in base/NSMapTable --- --- Running tests in base/NSMethodSignature --- --- Running tests in base/NSMutableArray --- --- Running tests in base/NSMutableAttributedString --- --- Running tests in base/NSMutableCharacterSet --- --- Running tests in base/NSMutableData --- --- Running tests in base/NSMutableDictionary --- --- Running tests in base/NSMutableIndexSet --- --- Running tests in base/NSMutableSet --- --- Running tests in base/NSMutableString --- --- Running tests in base/NSNotification --- --- Running tests in base/NSNumber --- --- Running tests in base/NSNumberFormatter --- --- Running tests in base/NSObject --- base/NSObject/initialize.m: Skipped set: initialize.m 201 ... +initialize runs concurrently. No SIGALRM present, this means we cannot stop the test on deadlock. --- Running tests in base/NSOperation --- --- Running tests in base/NSOrderedSet --- --- Running tests in base/NSPersonNameComponentsFormatter --- --- Running tests in base/NSPointerArray --- --- Running tests in base/NSPredicate --- --- Running tests in base/NSProcessInfo --- --- Running tests in base/NSProgress --- --- Running tests in base/NSProxy --- --- Running tests in base/NSRegularExpression --- --- Running tests in base/NSRunLoop --- --- Running tests in base/NSScanner --- --- Running tests in base/NSSet --- --- Running tests in base/NSSortDescriptor --- --- Running tests in base/NSStream --- base/NSStream/socket.m: Skipped set: socket.m 147 ... NSStream SSL functions not supported --- Running tests in base/NSString --- base/NSString/tilde.m: Skipped set: tilde.m 36 ... multiple slashes can't be removed on Windows Skipped set: tilde.m 59 ... tilde abbreviation test not implemented on Windows --- Running tests in base/NSTask --- --- Running tests in base/NSThread --- --- Running tests in base/NSTimeZone --- --- Running tests in base/NSUbiquitousKeyValueStore --- --- Running tests in base/NSUndoManager --- --- Running tests in base/NSURL --- base/NSURL/test00.m: Failed file: test00.m aborted without running all tests! --- Running tests in base/NSURLConnection --- base/NSURLConnection/test03.m: Failed file: test03.m aborted without running all tests! base/NSURLConnection/test04.m: Failed file: test04.m aborted without running all tests! --- Running tests in base/NSURLHandle --- --- Running tests in base/NSURLProtocol --- --- Running tests in base/NSURLRequest --- --- Running tests in base/NSURLSession --- base/NSURLSession/test02.m: Failed test: (2023-12-03 00:47:49.061 +0100) test02.m:92 ... request did not error Failed test: (2023-12-03 00:47:49.061 +0100) test02.m:95 ... request returned text base/NSURLSession/test03.m: Failed test: (2023-12-03 00:47:50.641 +0100) test03.m:80 ... request did not error Failed test: (2023-12-03 00:47:50.641 +0100) test03.m:89 ... request returned text --- Running tests in base/NSUserDefaults --- --- Running tests in base/NSUUID --- --- Running tests in base/NSValueTransformer --- --- Running tests in base/NSXMLDocument --- --- Running tests in base/NSXMLElement --- --- Running tests in base/NSXMLNode --- --- Running tests in base/NSXMLParser --- --- Running tests in base/PropertyLists --- --- Running tests in base/Unicode --- 8299 Passed tests 65 Dashed hopes 8 Skipped sets 4 Failed tests 3 Failed files Some testing was abandoned when a test program aborted. This is generally a severe problem and may mean that the package is completely unusable. You need to try to fix this and, if it is not due to some problem on your system, please help by submitting a patch (or at least a bug report) to the package maintainer. One or more tests failed. None of them should have. Please submit a patch to fix the problem or send a bug report to the package maintainer. Please see /c/tools-windows-msvc/src/gnustep-base/Tests/tests.log for more detail. ```

triplef commented 11 months ago

Awesome! I’ll try to set up my WoA hardware this week to give this a try, as well as a self-hosted runner for CI.

triplef commented 11 months ago

Thanks for the PR! :) I tried building this on a Windows ARM machine in a "vcvarsall arm64" prompt but got this error – any idea? Feel free to ping me on Slack if you have time to take a look together. 🙏

###### libobjc2 ######

### Cleaning project
HEAD is now at ab23f14 Initial Windows on ARM (AArch64) Support (#249)

### Updating project
Already up to date.
Synchronizing submodule url for 'third_party/robin-map'

### Running cmake
-- The C compiler identification is Clang 17.0.5 with GNU-like command-line
-- The ASM compiler identification is Clang with GNU-like command-line
-- Found assembler: C:/Program Files/LLVM/bin/clang.exe
-- The CXX compiler identification is Clang 17.0.5 with GNU-like command-line
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - failed
-- Check for working C compiler: C:/Program Files/LLVM/bin/clang.exe
-- Check for working C compiler: C:/Program Files/LLVM/bin/clang.exe - broken
CMake Error at C:/Program Files/CMake/share/cmake-3.27/Modules/CMakeTestCCompiler.cmake:67 (message):
  The C compiler

    "C:/Program Files/LLVM/bin/clang.exe"

  is not able to compile a simple test program.

  It fails with the following output:

    Change Dir: 'C:/Users/frede/Dev/djayApp/External/GNUstep/tools-windows-msvc/src/libobjc2/build-arm64-Debug/CMakeFiles/CMakeScratch/TryCompile-5d9duk'

    Run Build Command(s): C:/ProgramData/chocolatey/bin/ninja.exe -v cmTC_8aefb
    [1/2] C:\PROGRA~1\LLVM\bin\clang.exe  -m64   -m64  -O0 -g -Xclang -gcodeview -D_DEBUG -D_DLL -D_MT -Xclang --dependent-lib=msvcrtd -MD -MT CMakeFiles/cmTC_8aefb.dir/testCCompiler.c.obj -MF CMakeFiles\cmTC_8aefb.dir\testCCompiler.c.obj.d -o CMakeFiles/cmTC_8aefb.dir/testCCompiler.c.obj -c C:/Users/frede/Dev/djayApp/External/GNUstep/tools-windows-msvc/src/libobjc2/build-arm64-Debug/CMakeFiles/CMakeScratch/TryCompile-5d9duk/testCCompiler.c
    [2/2] cmd.exe /C "cd . && C:\PROGRA~1\LLVM\bin\clang.exe  -m64 -fuse-ld=lld-link -nostartfiles -nostdlib -m64  -O0 -g -Xclang -gcodeview -D_DEBUG -D_DLL -D_MT -Xclang --dependent-lib=msvcrtd -fuse-ld=lld   -Xlinker /subsystem:console CMakeFiles/cmTC_8aefb.dir/testCCompiler.c.obj -o cmTC_8aefb.exe -Xlinker /MANIFEST:EMBED -Xlinker /implib:cmTC_8aefb.lib -Xlinker /pdb:cmTC_8aefb.pdb -Xlinker /version:0.0   -lkernel32 -luser32 -lgdi32 -lwinspool -lshell32 -lole32 -loleaut32 -luuid -lcomdlg32 -ladvapi32 -loldnames  && cd ."
    FAILED: cmTC_8aefb.exe
    cmd.exe /C "cd . && C:\PROGRA~1\LLVM\bin\clang.exe  -m64 -fuse-ld=lld-link -nostartfiles -nostdlib -m64  -O0 -g -Xclang -gcodeview -D_DEBUG -D_DLL -D_MT -Xclang --dependent-lib=msvcrtd -fuse-ld=lld   -Xlinker /subsystem:console CMakeFiles/cmTC_8aefb.dir/testCCompiler.c.obj -o cmTC_8aefb.exe -Xlinker /MANIFEST:EMBED -Xlinker /implib:cmTC_8aefb.lib -Xlinker /pdb:cmTC_8aefb.pdb -Xlinker /version:0.0   -lkernel32 -luser32 -lgdi32 -lwinspool -lshell32 -lole32 -loleaut32 -luuid -lcomdlg32 -ladvapi32 -loldnames  && cd ."
    lld-link: error: could not open 'oldnames.lib': no such file or directory
    lld-link: error: could not open 'msvcrtd.lib': no such file or directory
    clang: error: linker command failed with exit code 1 (use -v to see invocation)
    ninja: build stopped: subcommand failed.

Clang:

clang version 17.0.5
Target: aarch64-pc-windows-msvc
Thread model: posix
InstalledDir: C:\Program Files\LLVM\bin

These are the components I installed in Visual Studio:

hmelder commented 11 months ago

That is weird. Can you provide the CMakeFiles/CMakeConfigureLog.yaml which is located in the CMake build directory?

What CMake version do you have? Is it the native ARM64 version, the VS version, or an emulated x64 release?

On my machine:

CMake 3.27.9 ARM64: https://github.com/Kitware/CMake/releases/download/v3.27.9/cmake-3.27.9-windows-arm64.msi
LLVM 17.0.5 woa64

C:\tools-windows-msvc>clang --version
clang version 17.0.5
Target: aarch64-pc-windows-msvc
Thread model: posix
InstalledDir: C:\Program Files\LLVM-woa64\bin

Feel free to ping me on Slack if you have time to take a look together

Alright!

triplef commented 11 months ago

I installed the arm64 version of CMake (I had the x64 one from Chocolatey), but that didn’t make a difference.

But I found out that the error is reproducible building a trivial C program, and that my LIB/LIBPATH environment variables are missing the path containing msvcrtd.lib et al. This is what I have in an "vcvarsall arm64" prompt:

LIB=C:\Program Files (x86)\Windows Kits\10\lib\10.0.22000.0\ucrt\arm64;C:\Program Files (x86)\Windows Kits\10\\lib\10.0.22000.0\\um\arm64
LIBPATH=C:\Program Files\Microsoft Visual Studio\2022\Professional\VC\Tools\MSVC\14.38.33130\lib\x86\store\references;C:\Program Files (x86)\Windows Kits\10\UnionMetadata\10.0.22000.0;C:\Program Files (x86)\Windows Kits\10\References\10.0.22000.0;C:\Windows\Microsoft.NET\Framework64\v4.0.30319

Whereas in a "vcvarsall x64" prompt I get this, which adds the <VS>\VC\Tools\MSVC\14.38.33130\lib\x64 directory containing the library:

LIB=C:\Program Files\Microsoft Visual Studio\2022\Professional\VC\Tools\MSVC\14.38.33130\ATLMFC\lib\x64;C:\Program Files\Microsoft Visual Studio\2022\Professional\VC\Tools\MSVC\14.38.33130\lib\x64;C:\Program Files (x86)\Windows Kits\10\lib\10.0.22000.0\ucrt\x64;C:\Program Files (x86)\Windows Kits\10\\lib\10.0.22000.0\\um\x64
LIBPATH=C:\Program Files\Microsoft Visual Studio\2022\Professional\VC\Tools\MSVC\14.38.33130\ATLMFC\lib\x64;C:\Program Files\Microsoft Visual Studio\2022\Professional\VC\Tools\MSVC\14.38.33130\lib\x64;C:\Program Files\Microsoft Visual Studio\2022\Professional\VC\Tools\MSVC\14.38.33130\lib\x86\store\references;C:\Program Files (x86)\Windows Kits\10\UnionMetadata\10.0.22000.0;C:\Program Files (x86)\Windows Kits\10\References\10.0.22000.0;C:\Windows\Microsoft.NET\Framework64\v4.0.30319

However if I manually add <VS>\VC\Tools\MSVC\14.38.33130\lib\arm to LIB and LIBPATH I get this trying to build a trivial C program:

p>clang test.c -o test.exe
lld-link: error: libcmt.lib(exe_main.obj): machine type arm conflicts with arm64
lld-link: error: libcmt.lib(loadcfg.obj): machine type arm conflicts with arm64
lld-link: error: libcmt.lib(utility.obj): machine type arm conflicts with arm64
lld-link: error: libcmt.lib(gs_support.obj): machine type arm conflicts with arm64
lld-link: error: libcmt.lib(matherr.obj): machine type arm conflicts with arm64
lld-link: error: libcmt.lib(argv_mode.obj): machine type arm conflicts with arm64
lld-link: error: libcmt.lib(commit_mode.obj): machine type arm conflicts with arm64
lld-link: error: libcmt.lib(file_mode.obj): machine type arm conflicts with arm64
lld-link: error: libcmt.lib(new_mode.obj): machine type arm conflicts with arm64
lld-link: error: libcmt.lib(thread_locale.obj): machine type arm conflicts with arm64
lld-link: error: libcmt.lib(tncleanup.obj): machine type arm conflicts with arm64
lld-link: error: libcmt.lib(env_mode.obj): machine type arm conflicts with arm64
lld-link: error: libcmt.lib(invalid_parameter_handler.obj): machine type arm conflicts with arm64
lld-link: error: libcmt.lib(denormal_control.obj): machine type arm conflicts with arm64
lld-link: error: libcmt.lib(default_local_stdio_options.obj): machine type arm conflicts with arm64
lld-link: error: libcmt.lib(matherr_detection.obj): machine type arm conflicts with arm64
lld-link: error: libcmt.lib(dyn_tls_init.obj): machine type arm conflicts with arm64
lld-link: error: libcmt.lib(dyn_tls_dtor.obj): machine type arm conflicts with arm64
lld-link: error: libcmt.lib(utility_desktop.obj): machine type arm conflicts with arm64
lld-link: error: libcmt.lib(initsect.obj): machine type arm conflicts with arm64
lld-link: error: too many errors emitted, stopping now (use /errorlimit:0 to see all errors)
clang: error: linker command failed with exit code 1 (use -v to see invocation)

Maybe interestingly when I build in a "x64 native tools command prompt" (aka vcvarsall x64) I get machine type x64 conflicts with arm64.

triplef commented 11 months ago

The fine folks at Linaro helped me figure out that I had incorrectly installed the build tools and ATL for (32-bit) "ARM" instead of "ARM64/ARM64EC".

The build is now running with these VC components – I’ll report back.

hmelder commented 11 months ago

The fine folks at Linaro helped me figure out that I had incorrectly installed the build tools and ATL for (32-bit) "ARM" instead of "ARM64/ARM64EC".

The build is now running with these VC components – I’ll report back.

Weird. I do not have any ATL components installed and it works just fine. We are not using Active Template Library afaik.

triplef commented 11 months ago

Ahh that’s entirely possible that ATL is not required for the toolchain, I just installed it because I think our app needs them.

hmelder commented 11 months ago

Ahh that’s entirely possible that ATL is not required for the toolchain, I just installed it because I think our app needs them.

That makes sense :)

gnustep / tools-windows-msvc

Initial Windows on ARM64 SDK Support #34