`xmake run` fails to locate shared objects on `x64` architecture on Linux

[nasso]

Describe the bug

Getting error while loading shared libraries: xxx.so: cannot open shared object file: No such file or directory when running a binary depending on a shared library from a package. This only happens when --arch=x64 on Linux (I tested on the ubuntu:latest Docker image).

xmake.lua:

add_requires("openal-soft", {configs={shared=true}})

target("hello")
    set_kind("binary")
    set_languages("c++17")
    add_files("hello.cpp")
    add_packages("openal-soft")

hello.cpp:

#include <iostream>
#include <AL/al.h>

int main() {
    ALuint buf;

    alGenBuffers(0, &buf);
    std::cout << "made buffer: " << buf << std::endl;
    alDeleteBuffers(1, &buf);
    return 0;
}

xmake config --arch=x64 --verbose
xmake --verbose
xmake run --verbose hello

Expected behavior

hello world!

Error output

/tmp/build/linux/x64/debug/hello: error while loading shared libraries: libopenal.so.1: cannot open shared object file: No such file or directory
error: execv(/tmp/build/linux/x64/debug/hello) failed(127)

See also: https://github.com/nasso/ige/runs/3036753466?check_suite_focus=true

Related Environment

OpenAL is NOT installed on the system (otherwise the system would find openal, but it wouldn't be the one built by xmake). A Docker image such as ubuntu:latest can be used to guarantee a clean environment.

• xmake version: 2.5.5
• os: ubuntu
• target platform: same as host, x64

[waruqi]

  => install openal-soft 1.21.1 .. ok
configure
{
    ccache = true
    host = linux
    mode = release
    ndk_stdcxx = true
    buildir = build
    arch = x86_64
    clean = true
    plat = linux
    kind = static
}
ruki@a34f4d48859d:/tmp/tes$ xmake -rv
checking for the linker (ld) ... g++
checking for gcc ... /usr/bin/gcc
checking for the c++ compiler (cxx) ... gcc
[ 25%]: ccache compiling.release hello.cpp
/usr/bin/ccache /usr/bin/gcc -c -m64 -std=c++17 -isystem /home/ruki/.xmake/packages/o/openal-soft/1.21.1/3cf33ddcb09c4d7fabe73225990b1de8/include -o build/.objs/hello/linux/x86_64/release/hello.cpp.o hello.cpp
checking for flags (-MMD -MF) ... ok
checking for flags (-fdiagnostics-color=always) ... ok
[ 50%]: linking.release hello
/usr/bin/g++ -o build/linux/x86_64/release/hello build/.objs/hello/linux/x86_64/release/hello.cpp.o -m64 -L/home/ruki/.xmake/packages/o/openal-soft/1.21.1/3cf33ddcb09c4d7fabe73225990b1de8/lib -lopenal -ldl -lpthread
[100%]: build ok!
ruki@a34f4d48859d:/tmp/tes$ xmake r
made buffer: 32765
ruki@a34f4d48859d:/tmp/tes$ xmake run -v hello
made buffer: 32764

It works for me on my ubuntu21.04

[nasso]

it looks like you didn't configure with --arch=x64 (not x86_64)

root@b21aa7fe7d6e:/tmp# xmake config --arch=x64 --verbose
configure
{
    ccache = true
    host = linux
    arch = x64
    buildir = build
    plat = linux
    kind = static
    mode = debug
    ndk_stdcxx = true
}
root@b21aa7fe7d6e:/tmp# xmake --verbose
[100%]: build ok!
root@b21aa7fe7d6e:/tmp# xmake run --verbose hello
/tmp/build/linux/x64/debug/hello: error while loading shared libraries: libopenal.so.1: cannot open shared object file: No such file or directory
error: execv(/tmp/build/linux/x64/debug/hello) failed(127)

[waruqi]

The linux platform of xmake does not have x64, you can only configure x86_64.

see xmake f --help

[nasso]

aah i see

in that case its probably better to warn or even better, error when an invalid architecture is used, this would have helped a lot troubleshooting my issue.

[SirLynix]

Is there a reason XMake doesn't handle x86_64 and x64 the same? It seems error prone

[waruqi]

Is there a reason XMake doesn't handle x86_64 and x64 the same? It seems error prone

This is because xmake will use self:is_arch(os.arch()) to determine whether the package is running in the host environment, and then automatically add package/installdir/lib to LD_LIBRARY_PATH

But os.arch() will only return x86_64, i386 or arm/arm64 (such as Raspberry Pi) on linux

https://github.com/xmake-io/xmake/blob/4d62362c12ec2c3dbf387cbee561574c051d4bcf/xmake/core/package/package.lua#L639-L645

[waruqi]

in that case its probably better to warn or even better, error when an invalid architecture is used, this would have helped a lot troubleshooting my issue.

You can set any other unknown arch, but xmake will treat it as a cross-compilation environment, so it will not automatically load the lib directory of the package to LD_LIBRARY_PATH, because xmake assumes that it cannot run these packages in the host environment.

[SirLynix]

But os.arch() will only return x86_64, i386 or arm/arm64 (such as Raspberry Pi) on linux

I understand that, it just feels weird that os.arch() will return x86_64 on Linux and x64 on Windows (is it a behavior of tbox?), or that "x86_64" and "x64" are not compatible in self:is_arch.

I think the current behavior is problematic and error-prone as "x64" on Linux will work except for some things that will silently break.

I think there are two solutions, either handle x86_64 and x64 exactly the same or treat x64 as an error on platforms returning x86_64. I would prefer the first, as it allow to select 64bits build without wondering if you're on Windows, Linux or macOS just by running xmake --arch=x86_64 or xmake --arch=x64.

Please note that this is only for x86 platforms, I think that x86_64 and ARM64 should not be handled the same, however I would expect xmake to trigger an error when using x86_64 arch on ARM (as it makes no sense).

[waruqi]

You can also run xmake f -p linux -a x86_64 --sdk=/xxxx on the arm host as a cross-compilation, so no error can be reported. It is also necessary to configure arm on the x86_64 host.

[SirLynix]

Cross compilation is a good point.

However it should be possible to uniformize or trigger an error on build (as the platform is known at this point).

So yeah the solution would be to preprocess/validate arch when configuring/building.

[waruqi]

However, the arch of the cross-compilation platform is unknown, and any unknown arch may be configured.

xmake f -p linux -a xxxarch --sdk=xxxx

Although xmake will not process these arch, it is possible for users to process it themselves.

if is_plat("xxxarch") then
    add_defines("HAVE_XXXARCH")
    -- ...
end

So these arch are still valid values

[SirLynix]

I get your point, but x64 and x86_64 are not unknown arch, they could be handled.

I get why it's not an easy problem, but the fact that os.arch() will return x64 on Windows and x84_64 on Linux, and that they are handled differently by XMake silently

[waruqi]

However, for xmake currently on linux, x64 is an unknown arch. Only xmake on windows will recognize it.

[SirLynix]

I still think this should be fixed somehow, maybe issue a warning when building with an unknown arch, or use x86 / x86_64 on Windows and Linux.

I understand why this occurs, but this is very error-prone and shouldn't be left as is imo.

[waruqi]

I don't plan to deal with it yet, and it is more appropriate to use x64 for windows. At present, the arch of xmake uses the platform's own naming convention. For example, arm64-v8a on android instead of arm64.

[SirLynix]

I understand why you wish to keep the platform own naming convention as a default, however as of today xmake is able to compile a x64 arch on Linux but will silently ignore some features (like op issue), this is problematic.

If you wish to stick to the platform naming convention (ex: x86_64 on Linux), then other naming conventions (such as x64 on Linux) should either be handled the same (xmake knowing that x86_64 and x64 are the same on Linux) or not handled at all (with xmake failing to build because it doesn't recognize the arch and asking the user to use x86_64).

Since I'm just repeating myself I won't answer this thread anymore, I just hope I convinced you that xmake being silent about this is problematic.