Static linking using musl doesn't work

[vaibhavkulkarni]

Issue Env OS: Manjaro 21.2.0 Graal VM: 21.3.0 JDK Version: JDK 11 JAVA_HOME: Set to GRAALVM_HOME Musl Version: 11.2.1 20211120 (GCC) ARCH: AMD64

All the steps mentioned here are followed.

Steps to reproduce the issue

1. Compile a simple "Hello, World!" program below:

public class TestStaticLinking { public static void main(String args[]) { System.out.println("Hello, World!"); } }

$GRAALVM_HOME/bin/javac TestStaicLinking.java

2. Try to get the statically linked executable using: native-image --native-image-info --verbose --static --libc=musl TestStaticLinking
3. Observe the build fail with following error message

native-image --native-image-info --verbose --static --libc=musl TestStaticLinking
Executing [
/home/vaibhav/workspace/tools/graalvm-ce-java11-21.3.0/bin/java \
-XX:+UseParallelGC \
-XX:+UnlockExperimentalVMOptions \
-XX:+EnableJVMCI \
-Dtruffle.TrustAllTruffleRuntimeProviders=true \
-Dtruffle.TruffleRuntime=com.oracle.truffle.api.impl.DefaultTruffleRuntime \
-Dgraalvm.ForcePolyglotInvalid=true \
-Dgraalvm.locatorDisabled=true \
-Dsubstratevm.IgnoreGraalVersionCheck=true \
--add-exports=java.base/com.sun.crypto.provider=ALL-UNNAMED \
--add-exports=java.base/jdk.internal.event=ALL-UNNAMED \
--add-exports=java.base/jdk.internal.loader=ALL-UNNAMED \
--add-exports=java.base/jdk.internal.logger=ALL-UNNAMED \
--add-exports=java.base/jdk.internal.misc=ALL-UNNAMED \
--add-exports=java.base/jdk.internal.module=ALL-UNNAMED \
--add-exports=java.base/jdk.internal.org.objectweb.asm=ALL-UNNAMED \
--add-exports=java.base/jdk.internal.org.xml.sax.helpers=ALL-UNNAMED \
--add-exports=java.base/jdk.internal.perf=ALL-UNNAMED \
--add-exports=java.base/jdk.internal.ref=ALL-UNNAMED \
--add-exports=java.base/jdk.internal.reflect=ALL-UNNAMED \
--add-exports=java.base/jdk.internal.util.xml.impl=ALL-UNNAMED \
--add-exports=java.base/jdk.internal.util.xml=ALL-UNNAMED \
--add-exports=java.base/sun.invoke.util=ALL-UNNAMED \
--add-exports=java.base/sun.nio.ch=ALL-UNNAMED \
--add-exports=java.base/sun.reflect.annotation=ALL-UNNAMED \
--add-exports=java.base/sun.reflect.generics.reflectiveObjects=ALL-UNNAMED \
--add-exports=java.base/sun.reflect.generics.repository=ALL-UNNAMED \
--add-exports=java.base/sun.reflect.generics.tree=ALL-UNNAMED \
--add-exports=java.base/sun.security.jca=ALL-UNNAMED \
--add-exports=java.base/sun.security.provider=ALL-UNNAMED \
--add-exports=java.base/sun.security.util=ALL-UNNAMED \
--add-exports=java.base/sun.text.spi=ALL-UNNAMED \
--add-exports=java.base/sun.util.calendar=ALL-UNNAMED \
--add-exports=java.base/sun.util.locale.provider=ALL-UNNAMED \
--add-exports=java.base/sun.util.resources=ALL-UNNAMED \
--add-exports=java.xml.crypto/org.jcp.xml.dsig.internal.dom=ALL-UNNAMED \
--add-exports=jdk.internal.vm.ci/jdk.vm.ci.aarch64=ALL-UNNAMED \
--add-exports=jdk.internal.vm.ci/jdk.vm.ci.amd64=ALL-UNNAMED \
--add-exports=jdk.internal.vm.ci/jdk.vm.ci.code.site=ALL-UNNAMED \
--add-exports=jdk.internal.vm.ci/jdk.vm.ci.code.stack=ALL-UNNAMED \
--add-exports=jdk.internal.vm.ci/jdk.vm.ci.code=ALL-UNNAMED \
--add-exports=jdk.internal.vm.ci/jdk.vm.ci.common=ALL-UNNAMED \
--add-exports=jdk.internal.vm.ci/jdk.vm.ci.hotspot.aarch64=ALL-UNNAMED \
--add-exports=jdk.internal.vm.ci/jdk.vm.ci.hotspot.amd64=ALL-UNNAMED \
--add-exports=jdk.internal.vm.ci/jdk.vm.ci.hotspot.sparc=ALL-UNNAMED \
--add-exports=jdk.internal.vm.ci/jdk.vm.ci.hotspot=ALL-UNNAMED \
--add-exports=jdk.internal.vm.ci/jdk.vm.ci.meta=ALL-UNNAMED \
--add-exports=jdk.internal.vm.ci/jdk.vm.ci.runtime=ALL-UNNAMED \
--add-exports=jdk.internal.vm.ci/jdk.vm.ci.services=ALL-UNNAMED \
--add-exports=jdk.internal.vm.ci/jdk.vm.ci.sparc=ALL-UNNAMED \
--add-exports=jdk.jfr/jdk.jfr.events=ALL-UNNAMED \
--add-exports=jdk.jfr/jdk.jfr.internal.consumer=ALL-UNNAMED \
--add-exports=jdk.jfr/jdk.jfr.internal.handlers=ALL-UNNAMED \
--add-exports=jdk.jfr/jdk.jfr.internal.jfc=ALL-UNNAMED \
--add-exports=jdk.jfr/jdk.jfr.internal=ALL-UNNAMED \
-XX:+UseJVMCINativeLibrary \
-Xss10m \
-Xms1g \
-Xmx14g \
-Duser.country=US \
-Duser.language=en \
-Djava.awt.headless=true \
-Dorg.graalvm.version=21.3.0 \
-Dorg.graalvm.config=CE \
-Dcom.oracle.graalvm.isaot=true \
-Djava.system.class.loader=com.oracle.svm.hosted.NativeImageSystemClassLoader \
-Xshare:off \
-Djdk.internal.lambda.disableEagerInitialization=true \
-Djdk.internal.lambda.eagerlyInitialize=false \
-Djava.lang.invoke.InnerClassLambdaMetafactory.initializeLambdas=false \
-javaagent:/home/vaibhav/workspace/tools/graalvm-ce-java11-21.3.0/lib/svm/builder/svm.jar \
-cp \
/home/vaibhav/workspace/tools/graalvm-ce-java11-21.3.0/lib/svm/builder/objectfile.jar:/home/vaibhav/workspace/tools/graalvm-ce-java11-21.3.0/lib/svm/builder/javacpp-shadowed.jar:/home/vaibhav/workspace/tools/graalvm-ce-java11-21.3.0/lib/svm/builder/llvm-wrapper-shadowed.jar:/home/vaibhav/workspace/tools/graalvm-ce-java11-21.3.0/lib/svm/builder/svm.jar:/home/vaibhav/workspace/tools/graalvm-ce-java11-21.3.0/lib/svm/builder/pointsto.jar:/home/vaibhav/workspace/tools/graalvm-ce-java11-21.3.0/lib/svm/builder/llvm-platform-specific-shadowed.jar:/home/vaibhav/workspace/tools/graalvm-ce-java11-21.3.0/lib/svm/builder/svm-llvm.jar \
--module-path \
/home/vaibhav/workspace/tools/graalvm-ce-java11-21.3.0/lib/truffle/truffle-api.jar \
'com.oracle.svm.hosted.NativeImageGeneratorRunner$JDK9Plus' \
-watchpid \
43285 \
-imagecp \
/home/vaibhav/workspace/junk/graal-native-musl-static:/home/vaibhav/workspace/tools/graalvm-ce-java11-21.3.0/lib/svm/library-support.jar \
-H:Path=/home/vaibhav/workspace/junk/graal-native-musl-static \
-H:+DumpTargetInfo \
-H:+StaticExecutable \
-H:UseLibC=musl \
-H:CLibraryPath=/home/vaibhav/workspace/tools/graalvm-ce-java11-21.3.0/lib/svm/clibraries/linux-amd64 \
'-H:Class@explicit main-class=TestStaticLinking' \
'-H:Name@main-class lower case as image name=teststaticlinking'
]
[teststaticlinking:43304]    classlist:   1,564.06 ms,  0.96 GB
[teststaticlinking:43304]        (cap):     692.68 ms,  0.96 GB
[teststaticlinking:43304]        setup:   2,774.96 ms,  0.96 GB
# Printing compilation-target information to: /home/vaibhav/workspace/junk/graal-native-musl-static/reports/target_info_20211202_133252.txt
[teststaticlinking:43304]     (clinit):     272.51 ms,  1.70 GB
# Printing native-library information to: /home/vaibhav/workspace/junk/graal-native-musl-static/reports/native_library_info_20211202_133303.txt
[teststaticlinking:43304]   (typeflow):   5,078.48 ms,  1.70 GB
[teststaticlinking:43304]    (objects):   3,917.87 ms,  1.70 GB
[teststaticlinking:43304]   (features):   1,294.06 ms,  1.70 GB
[teststaticlinking:43304]     analysis:  10,943.75 ms,  1.70 GB
[teststaticlinking:43304]     universe:   1,324.39 ms,  1.70 GB
[teststaticlinking:43304]      (parse):     964.57 ms,  1.70 GB
[teststaticlinking:43304]     (inline):   1,433.52 ms,  2.22 GB
[teststaticlinking:43304]    (compile):  12,532.56 ms,  2.34 GB
[teststaticlinking:43304]      compile:  16,077.01 ms,  2.34 GB
[teststaticlinking:43304]        image:   1,362.83 ms,  2.34 GB
[teststaticlinking:43304]        write:     227.03 ms,  2.34 GB
Fatal error:java.lang.RuntimeException: There was an error linking the native image: Linker command exited with 1

Linker command executed:
/home/vaibhav/workspace/tools/x86_64-linux-musl-native/bin/x86_64-linux-musl-gcc -z noexecstack -Wl,--gc-sections -Wl,-x -o /home/vaibhav/workspace/junk/graal-native-musl-static/teststaticlinking teststaticlinking.o /home/vaibhav/workspace/tools/graalvm-ce-java11-21.3.0/lib/svm/clibraries/linux-amd64/liblibchelper.a /home/vaibhav/workspace/tools/graalvm-ce-java11-21.3.0/lib/static/linux-amd64/musl/libnet.a /home/vaibhav/workspace/tools/graalvm-ce-java11-21.3.0/lib/static/linux-amd64/musl/libnio.a /home/vaibhav/workspace/tools/graalvm-ce-java11-21.3.0/lib/static/linux-amd64/musl/libjava.a /home/vaibhav/workspace/tools/graalvm-ce-java11-21.3.0/lib/static/linux-amd64/musl/libfdlibm.a /home/vaibhav/workspace/tools/graalvm-ce-java11-21.3.0/lib/static/linux-amd64/musl/libzip.a /home/vaibhav/workspace/tools/graalvm-ce-java11-21.3.0/lib/svm/clibraries/linux-amd64/libjvm.a -static -v -L/tmp/SVM-15391331042885571254 -L/home/vaibhav/workspace/tools/graalvm-ce-java11-21.3.0/lib/static/linux-amd64/musl -L/home/vaibhav/workspace/tools/graalvm-ce-java11-21.3.0/lib/svm/clibraries/linux-amd64 -lpthread -ldl -lz -lrt

Linker command output:
Using built-in specs.
COLLECT_GCC=/home/vaibhav/workspace/tools/x86_64-linux-musl-native/bin/x86_64-linux-musl-gcc
COLLECT_LTO_WRAPPER=/home/vaibhav/workspace/tools/x86_64-linux-musl-native/bin/../libexec/gcc/x86_64-linux-musl/11.2.1/lto-wrapper
Target: x86_64-linux-musl
Configured with: ../src_gcc/configure --enable-languages=c,c++,fortran CC='x86_64-linux-musl-gcc -static --static' CXX='x86_64-linux-musl-g++ -static --static' FC='x86_64-linux-musl-gfortran -static --static' CFLAGS='-g0 -O2 -fno-align-functions -fno-align-jumps -fno-align-loops -fno-align-labels -Wno-error' CXXFLAGS='-g0 -O2 -fno-align-functions -fno-align-jumps -fno-align-loops -fno-align-labels -Wno-error' FFLAGS='-g0 -O2 -fno-align-functions -fno-align-jumps -fno-align-loops -fno-align-labels -Wno-error' LDFLAGS='-s -static --static' --enable-default-pie --enable-static-pie --disable-cet --disable-bootstrap --disable-assembly --disable-werror --target=x86_64-linux-musl --prefix= --libdir=/lib --disable-multilib --with-sysroot=/ --enable-tls --disable-libmudflap --disable-libsanitizer --disable-gnu-indirect-function --disable-libmpx --enable-initfini-array --enable-libstdcxx-time=rt --enable-deterministic-archives --enable-libstdcxx-time --enable-libquadmath --enable-libquadmath-support --disable-decimal-float --build=x86_64-pc-linux-muslxx --host=x86_64-linux-musl
Thread model: posix
Supported LTO compression algorithms: zlib
gcc version 11.2.1 20211120 (GCC) 
COMPILER_PATH=/home/vaibhav/workspace/tools/x86_64-linux-musl-native/bin/../libexec/gcc/x86_64-linux-musl/11.2.1/:/home/vaibhav/workspace/tools/x86_64-linux-musl-native/bin/../libexec/gcc/:/home/vaibhav/workspace/tools/x86_64-linux-musl-native/bin/../lib/gcc/x86_64-linux-musl/11.2.1/../../../../x86_64-linux-musl/bin/
LIBRARY_PATH=/home/vaibhav/workspace/tools/x86_64-linux-musl-native/bin/../lib/gcc/x86_64-linux-musl/11.2.1/:/home/vaibhav/workspace/tools/x86_64-linux-musl-native/bin/../lib/gcc/:/home/vaibhav/workspace/tools/x86_64-linux-musl-native/bin/../lib/gcc/x86_64-linux-musl/11.2.1/../../../../x86_64-linux-musl/lib/:/home/vaibhav/workspace/tools/x86_64-linux-musl-native/bin/../lib/gcc/x86_64-linux-musl/11.2.1/../../../:/home/vaibhav/workspace/tools/x86_64-linux-musl-native/bin/../lib/:/home/vaibhav/workspace/tools/x86_64-linux-musl-native/bin/../usr/lib/
COLLECT_GCC_OPTIONS='-z' 'noexecstack' '-o' '/home/vaibhav/workspace/junk/graal-native-musl-static/teststaticlinking' '-static' '-v' '-L/tmp/SVM-15391331042885571254' '-L/home/vaibhav/workspace/tools/graalvm-ce-java11-21.3.0/lib/static/linux-amd64/musl' '-L/home/vaibhav/workspace/tools/graalvm-ce-java11-21.3.0/lib/svm/clibraries/linux-amd64' '-mtune=generic' '-march=x86-64' '-dumpdir' '/home/vaibhav/workspace/junk/graal-native-musl-static/teststaticlinking.'
 /home/vaibhav/workspace/tools/x86_64-linux-musl-native/bin/../libexec/gcc/x86_64-linux-musl/11.2.1/collect2 --sysroot=/home/vaibhav/workspace/tools/x86_64-linux-musl-native/bin/../ --eh-frame-hdr -m elf_x86_64 -static -pie --no-dynamic-linker -z text -Bsymbolic -o /home/vaibhav/workspace/junk/graal-native-musl-static/teststaticlinking -z noexecstack /home/vaibhav/workspace/tools/x86_64-linux-musl-native/bin/../lib/gcc/x86_64-linux-musl/11.2.1/../../../rcrt1.o /home/vaibhav/workspace/tools/x86_64-linux-musl-native/bin/../lib/gcc/x86_64-linux-musl/11.2.1/../../../crti.o /home/vaibhav/workspace/tools/x86_64-linux-musl-native/bin/../lib/gcc/x86_64-linux-musl/11.2.1/crtbeginS.o -L/tmp/SVM-15391331042885571254 -L/home/vaibhav/workspace/tools/graalvm-ce-java11-21.3.0/lib/static/linux-amd64/musl -L/home/vaibhav/workspace/tools/graalvm-ce-java11-21.3.0/lib/svm/clibraries/linux-amd64 -L/home/vaibhav/workspace/tools/x86_64-linux-musl-native/bin/../lib/gcc/x86_64-linux-musl/11.2.1 -L/home/vaibhav/workspace/tools/x86_64-linux-musl-native/bin/../lib/gcc -L/home/vaibhav/workspace/tools/x86_64-linux-musl-native/bin/../lib/gcc/x86_64-linux-musl/11.2.1/../../../../x86_64-linux-musl/lib -L/home/vaibhav/workspace/tools/x86_64-linux-musl-native/bin/../lib/gcc/x86_64-linux-musl/11.2.1/../../.. -L/home/vaibhav/workspace/tools/x86_64-linux-musl-native/bin/../lib -L/home/vaibhav/workspace/tools/x86_64-linux-musl-native/bin/../usr/lib --gc-sections -x teststaticlinking.o /home/vaibhav/workspace/tools/graalvm-ce-java11-21.3.0/lib/svm/clibraries/linux-amd64/liblibchelper.a /home/vaibhav/workspace/tools/graalvm-ce-java11-21.3.0/lib/static/linux-amd64/musl/libnet.a /home/vaibhav/workspace/tools/graalvm-ce-java11-21.3.0/lib/static/linux-amd64/musl/libnio.a /home/vaibhav/workspace/tools/graalvm-ce-java11-21.3.0/lib/static/linux-amd64/musl/libjava.a /home/vaibhav/workspace/tools/graalvm-ce-java11-21.3.0/lib/static/linux-amd64/musl/libfdlibm.a /home/vaibhav/workspace/tools/graalvm-ce-java11-21.3.0/lib/static/linux-amd64/musl/libzip.a /home/vaibhav/workspace/tools/graalvm-ce-java11-21.3.0/lib/svm/clibraries/linux-amd64/libjvm.a -lpthread -ldl -lz -lrt --start-group -lgcc -lgcc_eh -lc --end-group /home/vaibhav/workspace/tools/x86_64-linux-musl-native/bin/../lib/gcc/x86_64-linux-musl/11.2.1/crtendS.o /home/vaibhav/workspace/tools/x86_64-linux-musl-native/bin/../lib/gcc/x86_64-linux-musl/11.2.1/../../../crtn.o
/home/vaibhav/workspace/tools/x86_64-linux-musl-native/bin/../lib/gcc/x86_64-linux-musl/11.2.1/../../../../x86_64-linux-musl/bin/ld: teststaticlinking.o: warning: relocation in read-only section `.svm_heap'
/home/vaibhav/workspace/tools/x86_64-linux-musl-native/bin/../lib/gcc/x86_64-linux-musl/11.2.1/../../../../x86_64-linux-musl/bin/ld: read-only segment has dynamic relocations
collect2: error: ld returned 1 exit status
    at com.oracle.svm.hosted.image.NativeImageViaCC.handleLinkerFailure(NativeImageViaCC.java:502)
    at com.oracle.svm.hosted.image.NativeImageViaCC.write(NativeImageViaCC.java:449)
    at com.oracle.svm.hosted.NativeImageGenerator.doRun(NativeImageGenerator.java:657)
    at com.oracle.svm.hosted.NativeImageGenerator.run(NativeImageGenerator.java:488)
    at com.oracle.svm.hosted.NativeImageGeneratorRunner.buildImage(NativeImageGeneratorRunner.java:403)
    at com.oracle.svm.hosted.NativeImageGeneratorRunner.build(NativeImageGeneratorRunner.java:569)
    at com.oracle.svm.hosted.NativeImageGeneratorRunner.main(NativeImageGeneratorRunner.java:122)
    at com.oracle.svm.hosted.NativeImageGeneratorRunner$JDK9Plus.main(NativeImageGeneratorRunner.java:599)
[teststaticlinking:43304]      [total]:  34,714.89 ms,  2.34 GB
# Printing build artifacts to: /home/vaibhav/workspace/junk/graal-native-musl-static/teststaticlinking.build_artifacts.txt
Error: Image build request failed with exit status 1
com.oracle.svm.driver.NativeImage$NativeImageError: Image build request failed with exit status 1
    at com.oracle.svm.driver.NativeImage.showError(NativeImage.java:1762)
    at com.oracle.svm.driver.NativeImage.build(NativeImage.java:1473)
    at com.oracle.svm.driver.NativeImage.performBuild(NativeImage.java:1434)
    at com.oracle.svm.driver.NativeImage.main(NativeImage.java:1421)

[rodrigar-mx]

Thanks for sharing the issue. We will take a look into it and get back to you.

[dainiusjocas]

I've faced a similar issue https://github.com/dainiusjocas/lucene-grep/runs/4463042596?check_suite_focus=true

[dainiusjocas]

The problem seems to be caused by some updates (on November 23rd) in the musl compiler tool-chain. I've tried to compile a native image with an older release of musl and it worked fine.

Here is my workaround https://github.com/dainiusjocas/lucene-grep/commit/87085077bcb7660268867d860f919686960c3ad8

[rodrigar-mx]

Thanks for sharing the workaround @dainiusjocas. I confirm it works. This issue has already been reported with the native image team.

[ArjanSchouten]

Had the same issue because in my Dockerfile I used the latest version of musl toolchain. I now use this version from musl.cc: https://more.musl.cc/10.2.1/x86_64-linux-musl/x86_64-linux-musl-native.tgz and it works!

[jabrena]

Hi,

Anyone from the team reported the issue with latest musl release?

• https://musl.cc/
• http://more.musl.cc/

[gradinac]

Hey everyone :) As @dainiusjocas mentioned, this problem started happening due to a change in the gcc musl toolchain build process that started treating relocations in a read-only section at runtime as an error rather than a compile time warning. I don't think we can report this issue to the musl toolchain developers - this was probably an intended change from their side.

The reason why it fails with native-image is because a read only section that gets placed in a read only segment at runtime, .svm_heap, has relocations that can only be resolved at program startup.

I don't see a trivial way to get rid of these relocations (I assume most of them come from method address relocations, for e.g. for the vtables).

As a workaround, it's probably best to use GCC 10.2.1 for cross-compiling to musl. Alternatively, there may be a ld option that can prevent the linker from failing due to these relocations, but I'm not sure which one it is exactly :/

Tracked internally by GR-24601