oracle / graal

GraalVM compiles Java applications into native executables that start instantly, scale fast, and use fewer compute resources 🚀
https://www.graalvm.org
Other
20.37k stars 1.63k forks source link

[java 16] gcc -v execution fails with permission denied #3673

Closed rmannibucau closed 9 months ago

rmannibucau commented 3 years ago

Describe the issue Native-image of graalvm 21.2.0.r16 does not work on ubuntu.

Steps to reproduce the issue Please include both build steps as well as run steps

  1. install the native dependencies (zlib etc)
  2. create a hello world main + configure native image
  3. run with the version mentionned

Error:

Error: Collecting native-compiler info with '/usr/bin/gcc -v' failed
Error: To prevent native-toolchain checking provide command-line option -H:-CheckToolchain
com.oracle.svm.core.util.UserError$UserException: Collecting native-compiler info with '/usr/bin/gcc -v' failed
To prevent native-toolchain checking provide command-line option -H:-CheckToolchain
    at com.oracle.svm.core.util.UserError.abort(UserError.java:139)
    at com.oracle.svm.hosted.c.codegen.CCompilerInvoker.addSkipCheckingInfo(CCompilerInvoker.java:103)
    at com.oracle.svm.hosted.c.codegen.CCompilerInvoker.<init>(CCompilerInvoker.java:71)
    at com.oracle.svm.hosted.c.codegen.CCompilerInvoker$LinuxCCompilerInvoker.<init>(CCompilerInvoker.java:209)
    at com.oracle.svm.hosted.c.codegen.CCompilerInvoker.create(CCompilerInvoker.java:79)
    at com.oracle.svm.hosted.NativeImageGenerator.setupNativeImage(NativeImageGenerator.java:883)
    at com.oracle.svm.hosted.NativeImageGenerator.doRun(NativeImageGenerator.java:530)
    at com.oracle.svm.hosted.NativeImageGenerator.run(NativeImageGenerator.java:491)
    at com.oracle.svm.hosted.NativeImageGeneratorRunner.buildImage(NativeImageGeneratorRunner.java:380)
    at com.oracle.svm.hosted.NativeImageGeneratorRunner.build(NativeImageGeneratorRunner.java:543)
    at com.oracle.svm.hosted.NativeImageGeneratorRunner.main(NativeImageGeneratorRunner.java:119)
    at com.oracle.svm.hosted.NativeImageGeneratorRunner$JDK9Plus.main(NativeImageGeneratorRunner.java:573)

But

$ /usr/bin/gcc -v
Using built-in specs.
COLLECT_GCC=/usr/bin/gcc
COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-linux-gnu/10/lto-wrapper
OFFLOAD_TARGET_NAMES=nvptx-none:amdgcn-amdhsa:hsa
OFFLOAD_TARGET_DEFAULT=1
Target: x86_64-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Ubuntu 10.3.0-1ubuntu1' --with-bugurl=file:///usr/share/doc/gcc-10/README.Bugs --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,m2 --prefix=/usr --with-gcc-major-version-only --program-suffix=-10 --program-prefix=x86_64-linux-gnu- --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --libdir=/usr/lib --enable-nls --enable-bootstrap --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --with-default-libstdcxx-abi=new --enable-gnu-unique-object --disable-vtable-verify --enable-plugin --enable-default-pie --with-system-zlib --enable-libphobos-checking=release --with-target-system-zlib=auto --enable-objc-gc=auto --enable-multiarch --disable-werror --with-arch-32=i686 --with-abi=m64 --with-multilib-list=m32,m64,mx32 --enable-multilib --with-tune=generic --enable-offload-targets=nvptx-none=/build/gcc-10-gDeRY6/gcc-10-10.3.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-10-gDeRY6/gcc-10-10.3.0/debian/tmp-gcn/usr,hsa --without-cuda-driver --enable-checking=release --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu --with-build-config=bootstrap-lto-lean --enable-link-mutex
Thread model: posix
Supported LTO compression algorithms: zlib zstd
gcc version 10.3.0 (Ubuntu 10.3.0-1ubuntu1)

Describe GraalVM and your environment:

oubidar-Abderrahim commented 3 years ago

Thank you for reporting this, we will take a look into it and get back to you

oubidar-Abderrahim commented 3 years ago

Hi, I tested with simple hello world and it work as expected on Ubutnu. Could you make sure that all the dependencies are installed correctly (build-essential libz-dev zlib1g-dev), also try redownloading the latest version and try again.

rmannibucau commented 3 years ago

All was already there and already tested to redownload graalvm or reinstall the related packages (which btw work for other unrelated stuff):

sudo apt-get installbuild-essential libz-dev zlib1g-devv
[sudo] Mot de passe de rmannibucau : 
Lecture des listes de paquets... Fait
Construction de l'arbre des dépendances... Fait
Lecture des informations d'état... Fait      
Note : sélection de « zlib1g-dev » au lieu de « libz-dev »
build-essential est déjà la version la plus récente (12.8ubuntu3).
zlib1g-dev est déjà la version la plus récente (1:1.2.11.dfsg-2ubuntu6).
0 mis à jour, 0 nouvellement installés, 0 à enlever et 145 non mis à jour.

Fun think is that if I run with svm getCompilerInfo() it works so can be something with native-image or so. If it helps here is the complete command I'm launching:

/home/rmannibucau/.m2/repository/org/apache/geronimo/arthur/cache/graal/21.2.0.r16/distribution_exploded/bin/native-image -classpath /home/rmannibucau/.m2/repository/org/apache/johnzon/johnzon-core/1.2.14/johnzon-core-1.2.14-jakarta.jar:/home/rmannibucau/.m2/repository/io/yupiik/logging/yupiik-logging-jul/1.0.1/yupiik-logging-jul-1.0.1-jakarta.jar:/home/rmannibucau/.m2/repository/org/apache/geronimo/specs/geronimo-json_1.1_spec/1.4/geronimo-json_1.1_spec-1.4-jakarta.jar:/home/rmannibucau/.m2/repository/org/apache/johnzon/johnzon-mapper/1.2.14/johnzon-mapper-1.2.14-jakarta.jar:/home/rmannibucau/.m2/repository/io/yupiik/uship/backbone-reflect/0.0.1/backbone-reflect-0.0.1.jar:/home/rmannibucau/.m2/repository/io/yupiik/batch/yupiik-batch-runtime/0.0.17/yupiik-batch-runtime-0.0.17.jar:/home/rmannibucau/.m2/repository/org/apache/johnzon/johnzon-jsonb/1.2.14/johnzon-jsonb-1.2.14-jakarta.jar:/home/rmannibucau/.m2/repository/io/yupiik/uship/backbone-johnzon/0.0.1/backbone-johnzon-0.0.1.jar:/home/rmannibucau/.m2/repository/org/apache/geronimo/specs/geronimo-jsonb_1.0_spec/1.3/geronimo-jsonb_1.0_spec-1.3-jakarta.jar:/opt/rmannibucau/dev/test/target/classes -H:MaxRuntimeCompileMethods=1000 -H:+EnforceMaxRuntimeCompileMethods -H:+AddAllCharsets -H:+ReportExceptionStackTraces --no-fallback --static --allow-incomplete-classpath --report-unsupported-elements-at-runtime --enable-all-security-services com.testMain /opt/rmannibucau/dev/test/target/test.graal.bin
[/opt/rmannibucau/dev/test/target/test.graal.bin:152877]    classlist:     796.59 ms,  0.96 GB
[/opt/rmannibucau/dev/test/target/test.graal.bin:152877]        setup:     466.48 ms,  0.96 GB
Error: Collecting native-compiler info with '/usr/bin/gcc -v' failed
Error: To prevent native-toolchain checking provide command-line option -H:-CheckToolchain
com.oracle.svm.core.util.UserError$UserException: Collecting native-compiler info with '/usr/bin/gcc -v' failed
To prevent native-toolchain checking provide command-line option -H:-CheckToolchain
    at com.oracle.svm.core.util.UserError.abort(UserError.java:139)
    at com.oracle.svm.hosted.c.codegen.CCompilerInvoker.addSkipCheckingInfo(CCompilerInvoker.java:103)
    at com.oracle.svm.hosted.c.codegen.CCompilerInvoker.<init>(CCompilerInvoker.java:71)
    at com.oracle.svm.hosted.c.codegen.CCompilerInvoker$LinuxCCompilerInvoker.<init>(CCompilerInvoker.java:209)
    at com.oracle.svm.hosted.c.codegen.CCompilerInvoker.create(CCompilerInvoker.java:79)
    at com.oracle.svm.hosted.NativeImageGenerator.setupNativeImage(NativeImageGenerator.java:883)
    at com.oracle.svm.hosted.NativeImageGenerator.doRun(NativeImageGenerator.java:530)
    at com.oracle.svm.hosted.NativeImageGenerator.run(NativeImageGenerator.java:491)
    at com.oracle.svm.hosted.NativeImageGeneratorRunner.buildImage(NativeImageGeneratorRunner.java:380)
    at com.oracle.svm.hosted.NativeImageGeneratorRunner.build(NativeImageGeneratorRunner.java:543)
    at com.oracle.svm.hosted.NativeImageGeneratorRunner.main(NativeImageGeneratorRunner.java:119)
    at com.oracle.svm.hosted.NativeImageGeneratorRunner$JDK9Plus.main(NativeImageGeneratorRunner.java:573)
[/opt/rmannibucau/dev/test/target/test.graal.bin:152877]      [total]:   1,295.88 ms,  0.96 GB
# Printing build artifacts to: /opt/rmannibucau/dev/test/target/test.graal.bin.build_artifacts.txt
Error: Image build request failed with exit status 1
/home/rmannibucau/.m2/repository/org/apache/geronimo/arthur/cache/graal/21.2.0.r16/distribution_exploded/bin/native-image --version
GraalVM 21.2.0 Java 16 CE (Java Version 16.0.2+7-jvmci-21.2-b08)

Edit: seems it comes from the old way to extract a distribution which got broken in some more recent version:

import org.apache.geronimo.arthur.impl.nativeimage.archive.Extractor;
import org.apache.geronimo.arthur.impl.nativeimage.installer.SdkmanGraalVMInstaller;
import org.apache.geronimo.arthur.impl.nativeimage.installer.SdkmanGraalVMInstallerConfiguration;

import java.nio.file.Path;
import java.nio.file.Paths;

// needs org.apache.geronimo.arthur:arthur-impl:1.0.2 or 1.0.3-SNAPSHOT
public class Tmp {
    public static void main(String[] args) {
        final Extractor extractor = new Extractor();
        final SdkmanGraalVMInstaller sdkmanGraalVMInstaller = new SdkmanGraalVMInstaller(SdkmanGraalVMInstallerConfiguration.builder()
                .offline(false)
                .inheritIO(true)
                .url("https://github.com/graalvm/graalvm-ce-builds/releases/download/vm-21.2.0/graalvm-ce-java16-linux-amd64-21.2.0.tar.gz")
                .version("21.2.0")
                .platform("linux-amd64")
                .gav("a:b:c")
                .workdir(Paths.get("/tmp/graalcache"))
                .resolver(gav -> {
                    throw new IllegalStateException(gav);
                })
                .installer((gav, file) -> {
                    System.out.println(file);
                    return file;
                })
                .extractor(extractor::unpack)
                .build());
        final Path install = sdkmanGraalVMInstaller.install();
        sdkmanGraalVMInstaller.installNativeImage();
        System.out.println(install);
    }
}

will try to review what changed and why it is broken now

edit2: diff with a sdkman flavor and a "manual" installation. Seems the native-image works this way but the diff between both installation returns nothing, not sure what can cause this regression. Any idea?

oubidar-Abderrahim commented 3 years ago

If I understood correctly, installing GraalVM manually works, but installing it using sdkman causes this issue? is that right? I actually went ahead and installed GraalVM 21.2.0-r16 using sdkman but I still cannot reproduce the issue, Is it possible to attach the debugger and see what happens at com/oracle/svm/hosted/c/codegen/CCompilerInvoker.java:372?

rmannibucau commented 3 years ago

@oubidar-Abderrahim no, using previous Tmp class does not work (was working for earlier releases) - you have maven dependency in comment containing the needed dependencies. I assume it can be permissions related, can try to look further but Im a bit surprise it pops up now and not in a major. I debugged the invoker and nothing shocking in the process builder nor env was seen.

edit: think i found the high level cause: some file miss some "right" perm, forcing the equivalent of a chmod 777 on all files makes it functional so guess it is just a matter of refining the exact needs so this issue can be solved if you dont' consider this change as a regression (i'm fine both ways). FYI, the fix: https://github.com/apache/geronimo-arthur/commit/7c20f750130c6cf9b027750a71999b327ff5f3b6