search

oracle / graal

GraalVM compiles Java applications into native executables that start instantly, scale fast, and use fewer compute resources 🚀
https://www.graalvm.org
Other
20.01k stars 1.6k forks source link

Native-image build is failing with GraalVM 21.0.1 and --libc=musl #7717

Closed munishchouhan closed 5 months ago

munishchouhan commented 8 months ago

Describe the issue Native-image build for our CLI application is failing with GraalVM 21.0.1 and --libc=musl

It's working with GraalVM with Java version 17.0.8 in our other application https://github.com/seqeralabs/tower-cli

Steps to reproduce the issue Please include both build steps as well as run steps

  1. Check the actions of this PR https://github.com/seqeralabs/wave-cli/pull/30

Describe GraalVM and your environment:

More details Add any other information about the problem here. Especially important are stack traces or log output. Feel free to link to gists or to screenshots if necessary.

   Run ./gradlew app:nativeCompile
> Task :buildSrc:extractPluginRequests UP-TO-DATE
> Task :buildSrc:generatePluginAdapters UP-TO-DATE
> Task :buildSrc:compileJava UP-TO-DATE
> Task :buildSrc:compileGroovy NO-SOURCE
> Task :buildSrc:compileGroovyPlugins UP-TO-DATE
> Task :buildSrc:pluginDescriptors UP-TO-DATE
> Task :buildSrc:processResources UP-TO-DATE
> Task :buildSrc:classes UP-TO-DATE
> Task :buildSrc:jar UP-TO-DATE
> Task :app:buildInfo
> Task :app:compileJava UP-TO-DATE
> Task :app:compileGroovy NO-SOURCE
> Task :app:processResources UP-TO-DATE
> Task :app:classes UP-TO-DATE
> Task :app:jar

> Task :app:generateResourcesConfigFile
[native-image-plugin] Resources configuration written into /home/runner/work/wave-cli/wave-cli/app/build/native/generated/generateResourcesConfigFile/resource-config.json

> Task :app:nativeCompile
[native-image-plugin] GraalVM Toolchain detection is enabled
[native-image-plugin] GraalVM uses toolchain detection. Selected:
[native-image-plugin]    - language version: 21
[native-image-plugin]    - vendor: Oracle
[native-image-plugin]    - runtime version: 21.0.1+12-jvmci-23.1-b19
[native-image-plugin] Native Image executable path: /opt/hostedtoolcache/graalvm-jdk-21_linux-x64_bin/21.0.0/x64/graalvm-jdk-21.0.1+12.1/lib/svm/bin/native-image
Warning: The option '-H:BuildOutputJSONFile=/tmp/native-image-build-output.json' is experimental and must be enabled via '-H:+UnlockExperimentalVMOptions' in the future.
Warning: Please re-evaluate whether any experimental option is required, and either remove or unlock it. The build output lists all active experimental options, including where they come from and possible alternatives. If you think an experimental option should be considered as stable, please file an issue.
========================================================================================================================
GraalVM Native Image: Generating 'wave' (executable)...
========================================================================================================================
For detailed information and explanations on the build output, visit:
https://github.com/oracle/graal/blob/master/docs/reference-manual/native-image/BuildOutput.md
------------------------------------------------------------------------------------------------------------------------

[1/[8](https://github.com/seqeralabs/wave-cli/actions/runs/6704169715/job/18216213505?pr=30#step:6:9)] Initializing...                                                                                    (0.0s @ 0.08GB)
------------------------------------------------------------------------------------------------------------------------
                        0.6s (10.7% of total time) in 22 GCs | Peak RSS: 0.41GB | CPU load: 1.87
------------------------------------------------------------------------------------------------------------------------
Produced artifacts:
 /home/runner/work/wave-cli/wave-cli/app/build/native/nativeCompile/svm_err_b_20231102T080610.530_pid2481.md (build_info)
========================================================================================================================
Failed generating 'wave' after 4.8s.

The build process encountered an unexpected error:

> com.oracle.svm.core.util.VMError$HostedError: Unable to run '/tmp/SVM-1112065[9](https://github.com/seqeralabs/wave-cli/actions/runs/6704169715/job/18216213505?pr=30#step:6:10)053864540669/RISCV64LibCHelperDirectives' to compute offsets in C data structures. Is it possible that your antivirus software interferes and puts the resulting file into quarantine?

Please inspect the generated error report at:
/home/runner/work/wave-cli/wave-cli/app/build/native/nativeCompile/svm_err_b_20231[10](https://github.com/seqeralabs/wave-cli/actions/runs/6704169715/job/18216213505?pr=30#step:6:11)2T080610.530_pid2481.md

If you are unable to resolve this problem, please file an issue with the error report at:
https://graalvm.org/support

> Task :app:nativeCompile FAILED
FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':app:nativeCompile'.
13 actionable tasks: 4 executed, 9 up-to-date
> Process 'command '/opt/hostedtoolcache/graalvm-jdk-21_linux-x64_bin/21.0.0/x64/graalvm-jdk-21.0.1+[12](https://github.com/seqeralabs/wave-cli/actions/runs/6704169715/job/18216213505?pr=30#step:6:13).1/bin/native-image'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
> Get more help at https://help.gradle.org./

BUILD FAILED in 8s
Error: Process completed with exit code 1.
medoussboug commented 8 months ago

Thank you for reaching out, can you please make sure to follow the following requirements

munishchouhan commented 7 months ago

I installed all the required software mentioned in https://www.graalvm.org/latest/reference-manual/native-image/guides/build-static-executables/ But still getting Error: Now I am trying on Os: Ubuntu-22.04 Arch: amd64 (x86_64) Java: 17 and 21

1. It appears as though libz:.a is missing. Please install it.

Linker command executed:
/usr/bin/x86_64-linux-musl-gcc -z noexecstack -Wl,--gc-sections -Wl,--version-script,/tmp/SVM-4697601272570030653/exported_symbols.list -Wl,-x -o /home/ubuntu/wave-cli/app/build/native/nativeCompile/wave wave.o /home/ubuntu/.sdkman/candidates/java/17.0.9-graalce/lib/svm/clibraries/linux-amd64/liblibchelper.a /home/ubuntu/.sdkman/candidates/java/17.0.9-graalce/lib/static/linux-amd64/musl/libnet.a /home/ubuntu/.sdkman/candidates/java/17.0.9-graalce/lib/static/linux-amd64/musl/libextnet.a /home/ubuntu/.sdkman/candidates/java/17.0.9-graalce/lib/static/linux-amd64/musl/libnio.a /home/ubuntu/.sdkman/candidates/java/17.0.9-graalce/lib/static/linux-amd64/musl/libjava.a /home/ubuntu/.sdkman/candidates/java/17.0.9-graalce/lib/static/linux-amd64/musl/libfdlibm.a /home/ubuntu/.sdkman/candidates/java/17.0.9-graalce/lib/static/linux-amd64/musl/libzip.a /home/ubuntu/.sdkman/candidates/java/17.0.9-graalce/lib/svm/clibraries/linux-amd64/libjvm.a -static -v -L/tmp/SVM-4697601272570030653 -L/home/ubuntu/.sdkman/candidates/java/17.0.9-graalce/lib/static/linux-amd64/musl -L/home/ubuntu/.sdkman/candidates/java/17.0.9-graalce/lib/svm/clibraries/linux-amd64 -lz -lpthread -ldl -lrt

Linker command output:
Using built-in specs.
Reading specs from /usr/lib/x86_64-linux-musl/musl-gcc.specs
rename spec cpp_options to old_cpp_options
COLLECT_GCC=x86_64-linux-gnu-gcc
COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-linux-gnu/11/lto-wrapper
OFFLOAD_TARGET_NAMES=nvptx-none:amdgcn-amdhsa
OFFLOAD_TARGET_DEFAULT=1
Target: x86_64-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Ubuntu 11.4.0-1ubuntu1~22.04' --with-bugurl=file:///usr/share/doc/gcc-11/README.Bugs --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,m2 --prefix=/usr --with-gcc-major-version-only --program-suffix=-11 --program-prefix=x86_64-linux-gnu- --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --libdir=/usr/lib --enable-nls --enable-bootstrap --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --with-default-libstdcxx-abi=new --enable-gnu-unique-object --disable-vtable-verify --enable-plugin --enable-default-pie --with-system-zlib --enable-libphobos-checking=release --with-target-system-zlib=auto --enable-objc-gc=auto --enable-multiarch --disable-werror --enable-cet --with-arch-32=i686 --with-abi=m64 --with-multilib-list=m32,m64,mx32 --enable-multilib --with-tune=generic --enable-offload-targets=nvptx-none=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-gcn/usr --without-cuda-driver --enable-checking=release --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu --with-build-config=bootstrap-lto-lean --enable-link-serialization=2
Thread model: posix
Supported LTO compression algorithms: zlib zstd
gcc version 11.4.0 (Ubuntu 11.4.0-1ubuntu1~22.04)
COMPILER_PATH=/usr/lib/gcc/x86_64-linux-gnu/11/:/usr/lib/gcc/x86_64-linux-gnu/11/:/usr/lib/gcc/x86_64-linux-gnu/:/usr/lib/gcc/x86_64-linux-gnu/11/:/usr/lib/gcc/x86_64-linux-gnu/
LIBRARY_PATH=/usr/lib/gcc/x86_64-linux-gnu/11/:/usr/lib/gcc/x86_64-linux-gnu/11/../../../x86_64-linux-gnu/:/usr/lib/gcc/x86_64-linux-gnu/11/../../../../lib/:/lib/x86_64-linux-gnu/:/lib/../lib/:/usr/lib/x86_64-linux-gnu/:/usr/lib/../lib/:/usr/lib/gcc/x86_64-linux-gnu/11/../../../:/lib/:/usr/lib/
COLLECT_GCC_OPTIONS='-z' 'noexecstack' '-o' '/home/ubuntu/wave-cli/app/build/native/nativeCompile/wave' '-static' '-v' '-L/tmp/SVM-4697601272570030653' '-L/home/ubuntu/.sdkman/candidates/java/17.0.9-graalce/lib/static/linux-amd64/musl' '-L/home/ubuntu/.sdkman/candidates/java/17.0.9-graalce/lib/svm/clibraries/linux-amd64' '-specs=/usr/lib/x86_64-linux-musl/musl-gcc.specs' '-mtune=generic' '-march=x86-64' '-dumpdir' '/home/ubuntu/wave-cli/app/build/native/nativeCompile/wave.'
 /usr/lib/gcc/x86_64-linux-gnu/11/collect2 -plugin /usr/lib/gcc/x86_64-linux-gnu/11/liblto_plugin.so -plugin-opt=/usr/lib/gcc/x86_64-linux-gnu/11/lto-wrapper -plugin-opt=-fresolution=/tmp/cc3O4qPW.res -plugin-opt=-pass-through=/usr/lib/gcc/x86_64-linux-gnu/11/libgcc.a -plugin-opt=-pass-through=/usr/lib/gcc/x86_64-linux-gnu/11/libgcc_eh.a -plugin-opt=-pass-through=-lc -dynamic-linker /lib/ld-musl-x86_64.so.1 -nostdlib -static -z relro -o /home/ubuntu/wave-cli/app/build/native/nativeCompile/wave -z noexecstack /usr/lib/x86_64-linux-musl/Scrt1.o /usr/lib/x86_64-linux-musl/crti.o /usr/lib/gcc/x86_64-linux-gnu/11/crtbeginS.o -L/tmp/SVM-4697601272570030653 -L/home/ubuntu/.sdkman/candidates/java/17.0.9-graalce/lib/static/linux-amd64/musl -L/home/ubuntu/.sdkman/candidates/java/17.0.9-graalce/lib/svm/clibraries/linux-amd64 -L/usr/lib/x86_64-linux-musl -L /usr/lib/gcc/x86_64-linux-gnu/11/. --gc-sections --version-script /tmp/SVM-4697601272570030653/exported_symbols.list -x wave.o /home/ubuntu/.sdkman/candidates/java/17.0.9-graalce/lib/svm/clibraries/linux-amd64/liblibchelper.a /home/ubuntu/.sdkman/candidates/java/17.0.9-graalce/lib/static/linux-amd64/musl/libnet.a /home/ubuntu/.sdkman/candidates/java/17.0.9-graalce/lib/static/linux-amd64/musl/libextnet.a /home/ubuntu/.sdkman/candidates/java/17.0.9-graalce/lib/static/linux-amd64/musl/libnio.a /home/ubuntu/.sdkman/candidates/java/17.0.9-graalce/lib/static/linux-amd64/musl/libjava.a /home/ubuntu/.sdkman/candidates/java/17.0.9-graalce/lib/static/linux-amd64/musl/libfdlibm.a /home/ubuntu/.sdkman/candidates/java/17.0.9-graalce/lib/static/linux-amd64/musl/libzip.a /home/ubuntu/.sdkman/candidates/java/17.0.9-graalce/lib/svm/clibraries/linux-amd64/libjvm.a -lz -lpthread -ldl -lrt --start-group /usr/lib/gcc/x86_64-linux-gnu/11/libgcc.a /usr/lib/gcc/x86_64-linux-gnu/11/libgcc_eh.a -lc --end-group /usr/lib/gcc/x86_64-linux-gnu/11/crtendS.o /usr/lib/x86_64-linux-musl/crtn.o
/usr/bin/ld: cannot find -lz: No such file or directory
collect2: error: ld returned 1 exit status

Please inspect the generated error report at:
/home/ubuntu/wave-cli/app/build/native/nativeCompile/svm_err_b_20231215T133317.359_pid5787.md

If you are unable to resolve this problem, please file an issue with the error report at:
https://graalvm.org/support
arixmkii commented 6 months ago

Had the same issue. To workaround it I had to build latest (as of writing 1.3) zlib static with musl and put into ~/.cache/coursier/jvm/graalvm-java21@21.0.1/lib/static/linux-amd64/musl, because I didn't figure how to configure the library search path for SBT based build. This is the file command output to confirm that it is statically build x86_64 ELF app. Tested it moving to another box, where we had problem with the dynamic due to different glibc before and it worked.

GraalVM community 21.0.1, because SBT and coursier are preconfigured to use community edition. Build host Ubuntu based. App tested on Ubuntu based and RHEL based systems.

app: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, with debug_info, not stripped
tobiaswx commented 2 months ago

Are there any plans to enable static linking with musl on ARM64? What are the current reasons for these limitations?

https://www.graalvm.org/22.0/reference-manual/native-image/ARM64/