search

xmake-io / xmake

🔥 A cross-platform build utility based on Lua
https://xmake.io
Apache License 2.0
9.87k stars 776 forks source link

xmake无法将交叉链参数正确传向使用autoconf的依赖包 #5530

Closed Redbeanw44602 closed 4 weeks ago

Redbeanw44602 commented 1 month ago

Xmake 版本

v2.9.4+20240729

操作系统版本和架构

Manjaro Linux

描述问题

我正在 x86_64 主机上编译到 aarch64，从 -v 的输出结果看，没有向 ./configure 传递交叉链信息和 --host 参数等（已在错误日志中标出）

configure 完成后，依赖包使用错误的工具链编译并链接，之后又使用正确的工具链进行 test，test 失败。

期待的结果

package.tools.autoconf 和 package.tools.cmake 模块都是可以支持 mingw/cross/iphoneos/android 等交叉编译平台和工具链的，xmake 会自动传递对应的工具链进去，用户不需要做任何其他事情。

摘自文档

因此这应该是一个Bug，希望作者能及时修复 :)

工程配置

add_requires('lame 3.100')

target("test")
    add_packages("lame")
    set_kind("binary")
    add_files("test.cpp")

附加信息和错误日志

输入：

xmake f -p linux -a arm64-v8a -m debug --cross="aarch64-linux-gnu-" -v

输出：

...
note: install or modify (m) these packages (pass -y to skip confirm)?
in xmake-repo:
  -> lame 3.100 
please input: y (y/n/m)

checking for ping ... /usr/bin/ping
pinging the host(downloads.sourceforge.net) ... 65535 ms
/usr/bin/tar -xf lame-3.100.tar.gz -C source.tmp
./configure --enable-nasm --enable-shared=no --with-pic --prefix=/home/username/.xmake/packages/l/lame/3.100/0bccc3af89224cd588fed2fa2505d91d --with-pic ###BUG
checking build system type... x86_64-unknown-linux-gnu
checking host system type... x86_64-unknown-linux-gnu ###BUG
...
/usr/lib/gcc/aarch64-linux-gnu/6.5.0/../../../../aarch64-linux-gnu/bin/ld: skipping incompatible /home/username/.xmake/packages/l/lame/3.100/0bccc3af89224cd588fed2fa2505d91d/lib/libmp3lame.a when searching for -lmp3lame
/usr/lib/gcc/aarch64-linux-gnu/6.5.0/../../../../aarch64-linux-gnu/bin/ld: cannot find -lmp3lame
collect2: error: ld returned 1 exit status
> checking for c includes(lame/lame.h)
> checking for c funcs(lame_encode_buffer)
> checking for c links(mp3lame)
> checking for c snippet(has_cfuncs)
error: execv(/usr/bin/aarch64-linux-gnu-g++ -o /tmp/.xmake1000/240828/_C8BC95B7C77948208374AC0B68BE1760.b /tmp/.xmake1000/240828/_C8BC95B7C77948208374AC0B68BE1760.o -L/home/username/.xmake/packages/l/lame/3.100/0bccc3af89224cd588fed2fa2505d91d/lib -lmp3lame) failed(1)
  => install lame 3.100 .. failed
error: install failed!
Issues-translate-bot commented 1 month ago

Bot detected the issue body's language is not English, translate it automatically.

Title: xmake cannot correctly pass cross-chain parameters to dependent packages using autoconf

waruqi commented 1 month ago

目前只有 xmake f -p cross 切到 cross 平台才会作为交叉编译透传工具链

Issues-translate-bot commented 1 month ago

Bot detected the issue body's language is not English, translate it automatically.

Currently, only xmake f -p cross switched to the cross platform will be used as a cross-compilation transparent transmission tool chain

Redbeanw44602 commented 1 month ago

目前 xrepo 对 cross 平台支持似乎不是特别好，我看了一下（用到的依赖包里）on_install 如果带有平台参数（linux/windows/mingw）几乎没有一个是考虑了 cross 平台的。

或许一个叫 cross 的平台本身让人有些费解？cmake 中使用交叉编译一般要传工具链、目标平台的系统类型和处理器架构，按理说 -p linux -a arm64-v8a --cross="aarch64-linux-gnu-" 已将需要使用的东西都表达了。（本人理解，如果有不对恳请指正）

另外在尝试安装使用 cmake 的依赖包时（-p linux）xmake 是会传递完整的交叉链信息的，如 yaml-cpp：

...
  => download https://github.com/jbeder/yaml-cpp/archive/refs/tags/0.8.0.tar.gz .. ok
checking for aarch64-linux-gnu-gcc ... /usr/bin/aarch64-linux-gnu-gcc
checking for the c compiler (cc) ... aarch64-linux-gnu-gcc
checking for aarch64-linux-gnu-gcc ... /usr/bin/aarch64-linux-gnu-gcc
checking for the c++ compiler (cxx) ... aarch64-linux-gnu-gcc
checking for aarch64-linux-gnu-gcc ... /usr/bin/aarch64-linux-gnu-gcc
checking for the assember (as) ... aarch64-linux-gnu-gcc
checking for aarch64-linux-gnu-gcc-ar ... /usr/bin/aarch64-linux-gnu-gcc-ar
checking for the static library archiver (ar) ... aarch64-linux-gnu-gcc-ar
checking for aarch64-linux-gnu-g++ ... /usr/bin/aarch64-linux-gnu-g++
checking for the linker (ld) ... aarch64-linux-gnu-g++
checking for aarch64-linux-gnu-gcc-ranlib ... /usr/bin/aarch64-linux-gnu-gcc-ranlib
checking for the archive index generator (ranlib) ... aarch64-linux-gnu-gcc-ranlib
checking for /usr/bin/aarch64-linux-gnu-gcc ... ok
checking for flags (-fPIC) ... ok
checking for flags (-fPIC) ... ok
checking for flags (-fPIC) ... ok
checking for /usr/bin/aarch64-linux-gnu-g++ ... ok
checking for flags (-fPIC) ... ok
checking for aarch64-linux-gnu-g++ ... /usr/bin/aarch64-linux-gnu-g++
checking for the shared library linker (sh) ... aarch64-linux-gnu-g++
checking for flags (-fPIC) ... ok
checking for cmake ... /usr/bin/cmake
/usr/bin/cmake -DYAML_CPP_BUILD_TESTS=OFF -DCMAKE_BUILD_TYPE=Release -DYAML_BUILD_SHARED_LIBS=OFF -DCMAKE_INSTALL_PREFIX=/home/username/.xmake/packages/y/yaml-cpp/0.8.0/5cee0c2bc67a480fba56397b7957ccd9 -DCMAKE_INSTALL_LIBDIR:PATH=lib -G "Unix Makefiles" -DCMAKE_FIND_USE_INSTALL_PREFIX=0 "-DCMAKE_CXX_LINK_EXECUTABLE=/usr/bin/aarch64-linux-gnu-g++ <FLAGS> <CMAKE_CXX_LINK_FLAGS> <LINK_FLAGS> <OBJECTS> -o <TARGET> <LINK_LIBRARIES>" -DCMAKE_FIND_USE_CMAKE_SYSTEM_PATH=0 -DHAVE_FLAG_SEARCH_PATHS_FIRST=0 -DCMAKE_FIND_ROOT_PATH_MODE_PACKAGE=BOTH -DCMAKE_C_COMPILER=/usr/bin/aarch64-linux-gnu-gcc -DCMAKE_OSX_SYSROOT= -DCMAKE_STATIC_LINKER_FLAGS= -DCMAKE_FIND_ROOT_PATH_MODE_PROGRAM=NEVER -DCMAKE_FIND_ROOT_PATH_MODE_LIBRARY=BOTH -DCMAKE_CXX_COMPILER=/usr/bin/aarch64-linux-gnu-g++ -DCMAKE_FIND_ROOT_PATH_MODE_INCLUDE=BOTH -DCMAKE_AR=/usr/bin/aarch64-linux-gnu-gcc-ar -DCMAKE_RANLIB=/usr/bin/aarch64-linux-gnu-gcc-ranlib -DCMAKE_ASM_COMPILER=/usr/bin/aarch64-linux-gnu-gcc -DCMAKE_SYSTEM_NAME=Linux "-DCMAKE_CXX_FLAGS_RELEASE= -O3 -DNDEBUG" -DCMAKE_STATIC_LINKER_FLAGS_RELEASE= -DCMAKE_SHARED_LINK
...
waruqi commented 1 month ago

目前 xrepo 对 cross 平台支持似乎不是特别好，我看了一下（用到的依赖包里）on_install 如果带有平台参数（linux/windows/mingw）几乎没有一个是考虑了 cross 平台的。

这话说的，支持 cross 的包，没有一千，几百个还是有的，可以到 https://xrepo.xmake.io/#/packages/cross 上面看，全是支持 cross 的。

不能光看 on_install 的参数。。on_install(function (package)) 这种没平台限制，全平台支持的包也都多了去了，都全平台了，cross 当然也支持。

没有的提 pr 过来，加上支持就行了，如果加了 cross 包还是编译不过不支持，那你设置成 linux + cross 也没意义，一样过不了。

或许一个叫 cross 的平台本身让人有些费解？cmake 中使用交叉编译一般要传工具链、目标平台的系统类型和处理器架构，按理说 -p linux -a arm64-v8a --cross="aarch64-linux-gnu-" 已将需要使用的东西都表达了。（本人理解，如果有不对恳请指正）

本身这么搞没问题，xmake 对 target 编译这样也支持交叉编译。。但是对包的处理，这会复杂化 on_install 。。

像 windows 这种在包构建中，都是平台跟 msvc 绑定的，才能最大化简化包的构建配置，并且 ci 的测试资源有限，runtimes/compiler/mode 各种组合下，一个平台的 ci 压测，都已经十几个 job 了。。再去切 toolchain ，就是多一个数量级，不光是包配置会更复杂难维护，ci 资源也不够。。

所以交叉编译，只能独立到 cross 平台，单独压测。单独维护 。 linux 目前算好的了，至少能让你传完整 toolchain, (--toolchain=, set_toolchain) 。。但是散装的 --cross=xx 肯定支持不了。

目前只有 cross 平台，xmake-repo 所有支持 cross 的包，都是有严格 ci 压测的，至少等保证你安装没问题。。

但是即使 linux 上支持 cross 包，目前也没法上 ci 的，包安装能否通过，全看你人品。这边也没人力资源，ci 资源去维护保证稳定性。

star-hengxing commented 1 month ago

xmake 定义的 cross 平台，语义上确实有点含糊，但 is_cross 还是明确区分了 host 和 arch。 https://github.com/xmake-io/xmake/blob/0cfd96928e1f56c4f9dd9a82662ec67313da58e2/xmake/core/base/private/is_cross.lua#L25-L51

Redbeanw44602 commented 1 month ago

不能光看 on_install 的参数。。on_install(function (package)) 这种没平台限制，全平台支持的包也都多了去了，都全平台了，cross 当然也支持。

当然，所以我限定了“带有平台参数”。问题是这样的带有平台参数的包很多，大多数又不考虑 cross ， -p linux 又可以编译😂...

我觉得测试交叉平台的包意义不是很大，交叉链也太多了，测试不完的... 还有的厂商定制交叉链...

如果说为了节约资源弄一个 cross 平台个人认为还是欠考虑了，github ci并行有限制，但是作业可以排队的。

还有就是目前的问题是 cmake 的依赖包会传递交叉链参数（散装的toolchain），但 autoconf 的依赖包不会传相应参数给 configure...

waruqi commented 1 month ago

当然，所以我限定了“带有平台参数”。问题是这样的带有平台参数的包很多，大多数又不考虑 cross ， -p linux 又可以编译😂...

所以，这种提个 pr 过来改进下包不就完事了么

我觉得测试交叉平台的包意义不是很大，交叉链也太多了，测试不完的... 还有的厂商定制交叉链...

ci 过一遍，只能能提前规避 90% 以上的问题，否则 issues 还不得打爆，我哪有时间天天处理这种问题。

如果说为了节约资源弄一个 cross 平台个人认为还是欠考虑了，github ci并行有限制，但是作业可以排队的。

不光是 ci 资源，还有人力资源。ci 跪了，还不是得维护者去修，就一个 cross 平台，我们都修的累死累活了。。特么要 linux 各种平台全开 ci job 跑。。那提交一个包。我们不得修的累死。

你看下现在的 pr 上的 ci ，一个 pr 都已经几十个 job 了，linux 下，还单独开了两， linux + gcc，linux + clang

有时候一个 job 跪，都要修一个礼拜才能解决。。这些都是需要时间人力的支撑的，不是你说多开个 ci 排队跑跑就完事了的。。跑出来要有人修啊。。

如果你有兴趣维护，专门处理新开的 ci job 上的所有 error 修复，那我不介意多开几个 cross 平台 ci 去测试。

waruqi commented 1 month ago

还有就是目前的问题是 cmake 的依赖包会传递交叉链参数（散装的toolchain），但 autoconf 的依赖包不会传相应参数给 configure...

包安装，散装参数一概不支持，请走 --toolchain 和 set_toolchain 切工具链。

Redbeanw44602 commented 1 month ago

最后确认一下，意思是 cmake 支持 --cross 提供的“散装参数”是bug，autoconf 不支持是正常的？

Redbeanw44602 commented 1 month ago

如果是对 autoconf 支持有问题而您比较忙的话，有兴趣提一个 pr 来帮忙修。

Issues-translate-bot commented 1 month ago

Bot detected the issue body's language is not English, translate it automatically.

At present, xrepo does not seem to have particularly good support for the cross platform. I looked at (in the dependency packages used) if on_install contains platform parameters (linux/windows/mingw), almost none of them consider the cross platform.

Having said that, there are not even a thousand packages that support cross, but hundreds. You can go to https://xrepo.xmake.io/#/packages/cross to see that they all support cross.

You can't just look at the parameters of on_install. . on_install(function (package)) has no platform restrictions, and there are many packages supported by all platforms. Of course, cross is also supported.

Perhaps a platform called cross itself is a bit confusing? When using cross-compilation in cmake, you generally need to pass the tool chain, system type and processor architecture of the target platform. It stands to reason that -p linux -a arm64-v8a --cross="aarch64-linux-gnu-" will already have everything you need to use. Expressed. (I understand, please correct me if I am wrong)

This is no problem in itself, xmake also supports cross-compilation for target compilation. . But for package processing, this complicates on_install. .

In package construction like Windows, the platform is bound to msvc, so that the package construction configuration can be simplified to the greatest extent, and CI testing resources are limited. Under various combinations of runtimes/compiler/mode, a platform can be tested for CI stress , there are already more than ten jobs. . If you cut the toolchain again, it will be an order of magnitude more. Not only will the package configuration be more complicated and difficult to maintain, but CI resources will also be insufficient. .

Therefore, cross compilation can only be done independently on the cross platform and tested separately. Maintained separately. Linux is pretty good at the moment, at least it allows you to transfer the complete toolchain, (--toolchain=, set_toolchain). . But bulk --cross=xx is definitely not supported.

Issues-translate-bot commented 1 month ago

Bot detected the issue body's language is not English, translate it automatically.

The cross platform defined by xmake is indeed a bit ambiguous in semantics, but is_cross still clearly distinguishes host and arch. https://github.com/xmake-io/xmake/blob/0cfd96928e1f56c4f9dd9a82662ec67313da58e2/xmake/core/base/private/is_cross.lua#L25-L51

Issues-translate-bot commented 1 month ago

Bot detected the issue body's language is not English, translate it automatically.

Don’t just look at the parameters of on_install. . on_install(function (package)) has no platform restrictions, and there are many more packages supported by all platforms. Of course, cross is also supported.

Of course, so I qualified "with platform parameters". The problem is that there are many such packages with platform parameters, and most of them do not consider cross, and -p linux can be compiled 😂...

I don’t think it’s very meaningful to test cross-platform packages. There are too many cross-chains and there are endless tests... Some manufacturers customize cross-chains...

If you want to build a cross platform to save resources, I personally think it is not considered. Github ci has restrictions on parallelism, but jobs can be queued.

Another problem is that the dependency package of cmake will pass cross-chain parameters (bulk toolchain), but the dependency package of autoconf will not pass the corresponding parameters to configure...

waruqi commented 1 month ago

最后确认一下，意思是 cmake 支持 --cross 提供的“散装参数”是bug，autoconf 不支持是正常的？

目前对包的安装，不管是 cmake,meson/autoconf ，我都没对散装 cross 进行支持，只支持 toolchain ，这里面考虑的东西太多了。

没那么简单

Issues-translate-bot commented 1 month ago

Bot detected the issue body's language is not English, translate it automatically.

Of course, so I qualified "with platform parameters". The problem is that there are many such packages with platform parameters, and most of them do not consider cross and can be compiled with -p linux😂...

So, wouldn’t it be enough to submit a PR to improve and then package it?

I don’t think it’s very meaningful to test cross-platform packages. There are too many cross-chains and there are endless tests... Some manufacturers customize cross-chains...

If you go through ci, you can only avoid more than 90% of the problems in advance, otherwise the issues will not be solved. How can I have time to deal with such problems every day?

If you want to build a cross platform in order to save resources, I personally think it is not considered. Github ci has restrictions on parallelism, but jobs can be queued.

Not just ci resources, but human resources. CI is on its knees. It doesn’t have to be fixed by the maintainers. It’s just a cross platform. We are all exhausted from repairing it. . In particular, you need to run ci job on all Linux platforms. . Then submit a package. We must not cultivate until we are exhausted.

Look at the ci on the current PR. One PR already has dozens of jobs. Under Linux, two separate ones are opened, linux + gcc, linux + clang.

Sometimes it takes a week of practice to resolve a job. . These all require the support of time and manpower. You can't just open more ci and line up and run. . If you run out, someone needs to fix it. .

If you are interested in maintenance and specifically handling all error repairs on newly opened ci jobs, then I don't mind opening a few more cross platform ci for testing.

Issues-translate-bot commented 1 month ago

Bot detected the issue body's language is not English, translate it automatically.

Another problem is that the dependency package of cmake will pass cross-chain parameters (bulk toolchain), but the dependency package of autoconf will not pass the corresponding parameters to configure...

Package installation and bulk parameters are not supported. Please use --toolchain and set_toolchain to switch the toolchain.

Issues-translate-bot commented 1 month ago

Bot detected the issue body's language is not English, translate it automatically.

Finally, to confirm, it means that cmake supports the "bulk parameters" provided by --cross. Is it a bug? Is it normal that autoconf does not support it?

Issues-translate-bot commented 1 month ago

Bot detected the issue body's language is not English, translate it automatically.

If there is a problem with autoconf support and you are busy, you are interested in raising a PR to help fix it.

Issues-translate-bot commented 1 month ago

Bot detected the issue body's language is not English, translate it automatically.

Finally, please confirm. Does it mean that cmake supports the "bulk parameters" provided by --cross? Is it a bug? Is it normal that autoconf does not support it?

For the current package installation, whether it is cmake, meson/autoconf, I do not support bulk cross, only toolchain. There are too many things to consider.

Not that simple

Redbeanw44602 commented 1 month ago

目前对包的安装，不管是 cmake,meson/autoconf ，我都没对散装 cross 进行支持，只支持 toolchain ，这里面考虑的东西太多了。

额... 看这里，cmake 是支持的。

Issues-translate-bot commented 1 month ago

Bot detected the issue body's language is not English, translate it automatically.

For the current package installation, whether it is cmake, meson/autoconf, I do not support bulk cross, only toolchain. There are too many things to consider.

Um... Look here, cmake is supported.

waruqi commented 1 month ago

目前对包的安装，不管是 cmake,meson/autoconf ，我都没对散装 cross 进行支持，只支持 toolchain ，这里面考虑的东西太多了。

额... 看这里，cmake 是支持的。

那是因为你指定了 -a arm64 ，正好触发到了 cross 平台，https://github.com/xmake-io/xmake/blob/87630c8cf33105cc8c000e73cb7b67a2decfe02d/xmake/modules/package/tools/cmake.lua#L906

但这并不是特意支持，只是碰巧切进去了而已。。如果你没设置 arch ，仅仅设置 --cross=, -ld 散装设置参数，照样不行。。

而且光这里进去，上面提到的那些点不处理，后期还会遇到各种问题。。不是说 碰巧偶尔几个包可以，就说明支持了。

Issues-translate-bot commented 1 month ago

Bot detected the issue body's language is not English, translate it automatically.

For the current package installation, whether it is cmake, meson/autoconf, I do not support bulk cross, only toolchain. There are too many things to consider.

Um... Look here, cmake is supported.

That's because you specified -a arm64, which happened to trigger the cross platform, https://github.com/xmake-io/xmake/blob/87630c8cf33105cc8c000e73cb7b67a2decfe02d/xmake/modules/package/tools/cmake.lua#L906

But this is not a deliberate support, it just happens to be cut in. . If you don't set arch and just set --cross=, -ld bulk setting parameters, it still won't work. .