search

xmake-io / xmake

🔥 A cross-platform build utility based on Lua
https://xmake.io
Apache License 2.0
9.87k stars 776 forks source link

xmake fails to find host macOS toolchain #5455

Closed SirLynix closed 1 month ago

SirLynix commented 1 month ago

Xmake Version

2.9.4 dev

Operating System Version and Architecture

macOS x86_64

Describe Bug

On macOS arm64 runner when building a x86_64 library and having a host library (for example for compiling shaders), xmake will fail to find the right toolchain to build the host library.

This worked right but broke about a week ago.

Also while writing this issue it appears x86 and arm64 have the same issue on Windows: https://github.com/SirLynix/xmake-macos-issue/actions/runs/10318714333/job/28565640114

Expected Behavior

That xmake compiles fine.

Project Configuration

add_rules("mode.debug", "mode.release")

add_requires("fmt")
add_requires("fmt~host", {host = true})

target("foo")
    add_files("src/bar.cpp")
    add_packages("fmt")

the target is irrelevant as the issue will happen when installing packages.

I made a simple project reproducing the bug here: https://github.com/SirLynix/xmake-macos-issue

Additional Information and Error Logs

When xmake compiles the x86_64 fmt it finds the right toolchain:

> /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang -c -Qunused-arguments -target x86_64-apple-macos14.5 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX14.5.sdk -std=c++14 -isystem /Users/runner/.xmake/packages/f/fmt/11.0.2/e31e3962694d48808200c131cb0e03cf/include -o /var/folders/m_/cksx93ys47x4621g0zbw_m4m0000gn/T/.xmake501/240809/_97C206EA19234E30843DEF0901134850.o /var/folders/m_/cksx93ys47x4621g0zbw_m4m0000gn/T/.xmake501/240809/_316471E98D124847A6F43E89D5D3E922.cpp
> /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang++ -o /var/folders/m_/cksx93ys47x4621g0zbw_m4m0000gn/T/.xmake501/240809/_97C206EA19234E30843DEF0901134850.b /var/folders/m_/cksx93ys47x4621g0zbw_m4m0000gn/T/.xmake501/240809/_97C206EA19234E30843DEF0901134850.o -target x86_64-apple-macos14.5 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX14.5.sdk -lz -L/Users/runner/.xmake/packages/f/fmt/11.0.2/e31e3962694d48808200c131cb0e03cf/lib -lfmt
> checking for c++ includes(fmt/format.h)
> checking for c++ links(fmt)
> checking for c++ snippet(test)
  => install fmt 11.0.2 .. ok

but when it tries to compile the host (arm64 version):

> /usr/bin/clang -c -Qunused-arguments -std=c++14 -isystem /Users/runner/.xmake/packages/f/fmt/11.0.2/722eb203ae6d4dd5addb1e04f7e269c1/include -o /var/folders/m_/cksx93ys47x4621g0zbw_m4m0000gn/T/.xmake501/240809/_1545F093D1E8433088DC3E126D73FA60.o /var/folders/m_/cksx93ys47x4621g0zbw_m4m0000gn/T/.xmake501/240809/_316471E98D124847A6F43E89D5D3E922.cpp
> /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang++ -o /var/folders/m_/cksx93ys47x4621g0zbw_m4m0000gn/T/.xmake501/240809/_1545F093D1E8433088DC3E126D73FA60.b /var/folders/m_/cksx93ys47x4621g0zbw_m4m0000gn/T/.xmake501/240809/_1545F093D1E8433088DC3E126D73FA60.o -lz -L/Users/runner/.xmake/packages/f/fmt/11.0.2/722eb203ae6d4dd5addb1e04f7e269c1/lib -lfmt
ld: library 'z' not found
clang: error: linker command failed with exit code 1 (use -v to see invocation)
> checking for c++ includes(fmt/format.h)
> checking for c++ links(fmt)
> checking for c++ snippet(test)
checkinfo: ...gramdir/core/sandbox/modules/import/core/tool/linker.lua:75: @programdir/core/sandbox/modules/os.lua:378: execv(/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang++ -o /var/folders/m_/cksx93ys47x4621g0zbw_m4m0000gn/T/.xmake501/240809/_1545F093D1E8433088DC3E126D73FA60.b /var/folders/m_/cksx93ys47x4621g0zbw_m4m0000gn/T/.xmake501/240809/_1545F093D1E8433088DC3E126D73FA60.o -lz -L/Users/runner/.xmake/packages/f/fmt/11.0.2/722eb203ae6d4dd5addb1e04f7e269c1/lib -lfmt) failed(1)

full logs

additionnally removing the -z ldflags and adding verbose linker results in 

> /usr/bin/clang -c -Qunused-arguments -std=c++14 -isystem /Users/runner/work/NazaraEngine/xmake-global/.xmake/packages/f/fmt/11.0.2/871df940863543fea595183887c1ce3e/include -o /var/folders/zn/hj183dg15s713b47j2wlhwzw0000gn/T/.xmake501/240805/_489B448B771F4510899A93D3A18B7350.o /var/folders/zn/hj183dg15s713b47j2wlhwzw0000gn/T/.xmake501/240805/_316471E98D124847A6F43E89D5D3E922.cpp
> /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang++ -o /var/folders/zn/hj183dg15s713b47j2wlhwzw0000gn/T/.xmake501/240805/_489B448B771F4510899A93D3A18B7350.b /var/folders/zn/hj183dg15s713b47j2wlhwzw0000gn/T/.xmake501/240805/_489B448B771F4510899A93D3A18B7350.o -Xlinker -v -L/Users/runner/work/NazaraEngine/xmake-global/.xmake/packages/f/fmt/11.0.2/871df940863543fea595183887c1ce3e/lib -lfmt
@(#)PROGRAM:ld PROJECT:ld-1053.12
BUILD 10:15:51 Mar 29 2024
configured to support archs: armv6 armv7 armv7s arm64 arm64e arm64_32 i386 x86_64 x86_64h armv6m armv7k armv7m armv7em
will use ld-classic for: armv6 armv7 armv7s arm64_32 i386 armv6m armv7k armv7m armv7em
LTO support using: LLVM version 15.0.0 (static support for 29, runtime is 29)
TAPI support using: Apple TAPI version 15.0.0 (tapi-1500.3.2.2)
Library search paths:
    /Users/runner/work/NazaraEngine/xmake-global/.xmake/packages/f/fmt/11.0.2/871df940863543fea595183887c1ce3e/lib
    /usr/lib
    /usr/lib/swift
Framework search paths:
    /Library/Frameworks
    /System/Library/Frameworks
ld: library 'c++' not found
clang: error: linker command failed with exit code 1 (use -v to see invocation)

https://github.com/NazaraEngine/NazaraEngine/actions/runs/10250822598/job/28359351809#step:11:31022

waruqi commented 1 month ago

I will look at it in these days.

waruqi commented 1 month ago

this patch will break it. https://github.com/xmake-io/xmake/commit/aa278bc7fc0b723a315120bb95531991b7939229

I will try to fix it in next days.

waruqi commented 1 month ago

I added a cli.bisect util to find patch. https://github.com/xmake-io/xmake/pull/5462

waruqi commented 1 month ago

and this patch will also break it. https://github.com/xmake-io/xmake-repo/blob/8822e12a8bd9a85bb34b13bacbe1a0899968c964/packages/f/fmt/xmake.lua#L56

I know the cause of the problem, it's supposed to be the toolchain check and load order that's causing the problem, but I need some more time to figure out how to fix it.

waruqi commented 1 month ago

fmt package issue has been fixed. https://github.com/xmake-io/xmake-repo/pull/4942

it should works now for your project. But I don't have a better way to get on_load to support has_tool, and some packages rely on it to set deps, and I haven't thought of a good way to get around that yet.