duplicate symbol of compiler_rt builtin functions

[yujincheng08]

mold: error: duplicate symbol: /lib/clang/18/lib/linux/libclang_rt.builtins-arm-android.a(divsi3.S.o): out/armeabi-v7a/librs.a(compiler_builtins-88872bbaad705f50.compiler_builtins.6894e964b64e61b2-cgu.0.rcgu.o): __divsi3
mold: error: duplicate symbol: /lib/clang/18/lib/linux/libclang_rt.builtins-arm-android.a(ashldi3.c.o): out/armeabi-v7a/librs.a(compiler_builtins-88872bbaad705f50.compiler_builtins.6894e964b64e61b2-cgu.0.rcgu.o): __ashldi3
mold: error: duplicate symbol: /lib/clang/18/lib/linux/libclang_rt.builtins-arm-android.a(udivsi3.S.o): out/armeabi-v7a/librs.a(compiler_builtins-88872bbaad705f50.compiler_builtins.6894e964b64e61b2-cgu.0.rcgu.o): __udivsi3
mold: error: duplicate symbol: /lib/clang/18/lib/linux/libclang_rt.builtins-arm-android.a(lshrdi3.c.o): out/armeabi-v7a/librs.a(compiler_builtins-88872bbaad705f50.compiler_builtins.6894e964b64e61b2-cgu.0.rcgu.o): __lshrdi3

two .a files: a.tar.gz

while lld compiles.

[rui314]

Can you re-link your program with -Wl,-repro and share the generated tar file? That option is explained here: https://github.com/rui314/mold/blob/main/docs/mold.md#mold-specific-options

[yujincheng08]

Hi, adding -Wl,-repo does not produce a tar file if this error happens (without this error mold does generate tar files).

[yujincheng08]

$ "/opt/homebrew/bin/ld.mold" -EL -z now -z relro -z max-page-size=4096 -X --hash-style=gnu --eh-frame-hdr -m armelf_linux_eabi -pie -dynamic-linker /system/bin/linker -o ./obj/local/armeabi-v7a/magisk /Users/shana/Library/Android/sdk/ndk/magisk/toolchains/llvm/prebuilt/darwin-x86_64/bin/../sysroot/usr/lib/arm-linux-androideabi/23/crtbegin_dynamic.o -L/Users/shana/Library/Android/sdk/ndk/magisk/toolchains/llvm/prebuilt/darwin-x86_64/lib/clang/18/lib/linux/arm -L/Users/shana/Library/Android/sdk/ndk/magisk/toolchains/llvm/prebuilt/darwin-x86_64/bin/../sysroot/usr/lib/arm-linux-androideabi/23 -L/Users/shana/Lib
rary/Android/sdk/ndk/magisk/toolchains/llvm/prebuilt/darwin-x86_64/bin/../sysroot/usr/lib/arm-linux-androideabi -L/Users/shana/Library/Android/sdk/ndk/magisk/toolchains/llvm/prebuilt/darwin-x86_64/bin/../sysroot/usr/lib/../lib -L/Users/shana/Library/Android/sdk/ndk/magisk/toolchains/llvm/prebuilt/darwin-x86_64/bin/../sysroot/usr/lib -plugin /Users/shana/Library/Android/sdk/ndk/magisk/toolchains/llvm/prebuilt/darwin-x86_64/bin/../lib/LLVMgold.dylib -plugin-opt=mcpu=generic -plugin-opt=-emulated-tls -rpath-link=/Users/shana/Library/Android/sdk/ndk/magisk/toolchains/llvm/prebuilt/darwin-x86_64/sysroot/usr/lib/arm-linux-androideabi/23 -rpath-link=./obj/local/armeabi-v7a ./obj/local/armeabi-v7a/objs/magisk/core/applets.o ./obj/local/armeabi-v7a/objs/magisk/core/magisk.o ./obj/local/armeabi-v7a/objs/magisk/core/daemon.o ./obj/local/armeabi-v7a/objs/magisk/core/bootstages.o ./obj/local/armeabi-v7a/objs/magisk/core/socket.o ./obj/local/armeabi-v7a/objs/magisk/core/db.o ./obj/local/armeabi-v7a/objs/magisk/core/package.o ./obj/local/armeabi-v7a/objs/magisk/core/scripting.o ./obj/local/armeabi-v7a/objs/magisk/core/selinux.o ./obj/local/armeabi-v7a/objs/magisk/core/module.o ./obj/local/armeabi-v7a/objs/magisk/core/thread.o ./obj/local/armeabi-v7a/objs/magisk/core/core-rs.o ./obj/local/armeabi-v7a/objs/magisk/core/resetprop/resetprop.o ./obj/local/armeabi-v7a/objs/magisk/core/su/su.o ./obj/local/armeabi-v7a/objs/magisk/core/su/connect.o ./obj/local/armeabi-v7a/objs/magisk/core/su/pts.o ./obj/local/armeabi-v7a/objs/magisk/core/su/su_daemon.o ./obj/local/armeabi-v7a/objs/magisk/core/zygisk/entry.o ./obj/local/armeabi-v7a/objs/magisk/core/zygisk/main.o ./obj/local/armeabi-v7a/objs/magisk/core/zygisk/module.o ./obj/local/armeabi-v7a/objs/magisk/core/zygisk/hook.o ./obj/local/armeabi-v7a/objs/magisk/core/deny/cli.o ./obj/local/armeabi-v7a/objs/magisk/core/deny/utils.o ./obj/local/armeabi-v7a/objs/magisk/core/deny/logcat.o ./obj/local/armeabi-v7a/libbase.a ./obj/local/armeabi-v7a/libsystemproperties.a ./obj/local/armeabi-v7a/liblsplt.a src/../out/armeabi-v7a/libmagisk-rs.a ./obj/local/armeabi-v7a/libcxx.a -latomic --gc-sections --build-id=sha1 --no-rosegment --icf=all -repro --dynamic-list=src/exported_sym.txt -z max-page-size=16384 --no-undefined --fatal-warnings --no-undefined-version -llog -lc -lm -lm /Users/shana/Library/Android/sdk/ndk/magisk/toolchains/llvm/prebuilt/darwin-x86_64/lib/clang/18/lib/linux/libclang_rt.builtins-arm-android.a -l:libunwind.a -ldl -lc /Users/shana/Library/Android/sdk/ndk/magisk/toolchains/llvm/prebuilt/darwin-x86_64/lib/clang/18/lib/linux/libclang_rt.builtins-arm-android.a -l:libunwind.a -ldl /Users/shana/Library/Android/sdk/ndk/magisk/toolchains/llvm/prebuilt/darwin-x86_64/bin/../sysroot/usr/lib/arm-linux-androideabi/23/crtend_android.o

mold: error: duplicate symbol: /Users/shana/Library/Android/sdk/ndk/magisk/toolchains/llvm/prebuilt/darwin-x86_64/lib/clang/18/lib/linux/libclang_rt.builtins-arm-android.a(divsi3.S.o): out/armeabi-v7a/libmagisk-rs.a(compiler_builtins-88872bbaad705f50.compiler_builtins.6894e964b64e61b2-cgu.0.rcgu.o): __divsi3
mold: error: duplicate symbol: /Users/shana/Library/Android/sdk/ndk/magisk/toolchains/llvm/prebuilt/darwin-x86_64/lib/clang/18/lib/linux/libclang_rt.builtins-arm-android.a(ashldi3.c.o): out/armeabi-v7a/libmagisk-rs.a(compiler_builtins-88872bbaad705f50.compiler_builtins.6894e964b64e61b2-cgu.0.rcgu.o): __ashldi3
mold: error: duplicate symbol: /Users/shana/Library/Android/sdk/ndk/magisk/toolchains/llvm/prebuilt/darwin-x86_64/lib/clang/18/lib/linux/libclang_rt.builtins-arm-android.a(udivsi3.S.o): out/armeabi-v7a/libmagisk-rs.a(compiler_builtins-88872bbaad705f50.compiler_builtins.6894e964b64e61b2-cgu.0.rcgu.o): __udivsi3
mold: error: duplicate symbol: /Users/shana/Library/Android/sdk/ndk/magisk/toolchains/llvm/prebuilt/darwin-x86_64/lib/clang/18/lib/linux/libclang_rt.builtins-arm-android.a(lshrdi3.c.o): out/armeabi-v7a/libmagisk-rs.a(compiler_builtins-88872bbaad705f50.compiler_builtins.6894e964b64e61b2-cgu.0.rcgu.o): __lshrdi3

$ ls obj/local/armeabi-v7a/magisk.tar             
ls: obj/local/armeabi-v7a/magisk.tar: No such file or directory

[rui314]

Can you try again with -Wl,--allow-multiple-definition?

[yujincheng08]

yes, it compiles with -Wl,-z,muldefs

[rui314]

Can you share the tar file?

[yujincheng08]

Sure, here you are: magisk.repro.tar.gz

[rui314]

libclang_rt.builtins-arm-android.a(divsi3.S.o) contains a strong definition of __aeabi_idiv while libmagisk-rs.a(compiler_builtins-ea0fdc2d15c27f56.compiler_builtins.46e4339db9da5858-cgu.0.rcgu.o) contains a weak definition of it. mold picks up the one in divsi3.S.o and pull out that file from the archive. However, the file also contains a definition of __divsi3 which caused the issue.

I don't think we can fix it without breaking existing programs. You may want to make a change to your program so that libmagisk-rs.a does not contain a definition of __aeabi_idiv. It looks actually odd to me that the library contains compiler builtin functions.

[yujincheng08]

libmagisk-rs.a is generated by rust's cargo and I have no idea how to avoid packing compiler builtins.

However, how does lld deal with this?

[rui314]

lld probably picks up the other file, but that is somewhat arbitrary. I'd say that the configuration is fragile and unnecessarily challenging for the linker. And I think packing compiler builtin functions into a user-provided library is a signal that something is not right with the library.

[yujincheng08]

After investigation, I found that libmagisk-rs.a's compiler_builtins is included by rust's -Zbuild-std and I cannot get rid of it. And the other one is automatically included by the compiler, which does not make sense to get rid of it.

Not an expert in linking, but why can't the linker ignore the weak definition of a symbol when there's a strong definition? One thing I usually do is to define some system functions in weak that may be missed in some building environments.

[rui314]

We do ignore weak definitions when there are strong ones. The issue here is that we have two strong definitions of __divsi3 and such.

[yujincheng08]

oh i misunderdtand. now i see whats going on.

[rui314]

It looks like -Z build-std is an experimental feature (https://doc.rust-lang.org/cargo/reference/unstable.html#build-std). You may want to file a bug against them.

As far as I can tell, if libclang_rt.builtins-arm-android.a does not contain a definition of __aeabi_idiv at all or contains a strong definition of it, no problem should occur. Currently, the file contains a weak definition of the symbol. Since mold prefers a strong definition when pulling out an object file, the file in libclang_rt.builtins-arm-android.a is not chosen to resolve that symbol. As a result, both libclang_rt.builtins-arm-android.a(divsi3.S.o) and librs.a(compiler_builtins-88872bbaad705f50.compiler_builtins.6894e964b64e61b2-cgu.0.rcgu.o) are pulled out, which results in a symbol conflict error.

[yujincheng08]

thanks

[rui314]

If you filed a bug against Rust, please copy-n-paste the URL here for tracking purpose.