Unable to use LLVM compiler backend on my aarch64 system

[Karm]

Hello,

Having this class:

class Main {
public static void main(String... args) {
System.out.println("Hi!");
}
}

and GraalVM b1a2f4c7 (LabsJDK 21+35-jvmci-23.1-b15-debug), built with:

mx --dynamicimports /substratevm,/tools,/sulong --exclude-components=nju,nic,ni,nil,llp --force-bash-launchers=polyglot build

I am unable to compile the class to a native image executable:

$ native-image --bundle-create -cp . -H:CompilerBackend=llvm -Dllvm.bin.dir=/usr/bin --native-compiler-path=/usr/bin/clang Main Main 2>&1 | tee log2.log
...
------------------------------------------------------------------------------------------------------------------------
[1/8] Initializing...                                                                                   (54.8s @ 0.21GB)
 Java version: 21+35, vendor version: GraalVM CE 21-dev+35.1
 Graal compiler: optimization level: 2, target machine: armv8-a
 C compiler: clang (llvm, aarch64, 16.0.6)
 Garbage collector: Serial GC (max heap size: 80% of RAM)
 1 user-specific feature(s):
 - com.oracle.svm.thirdparty.gson.GsonFeature
------------------------------------------------------------------------------------------------------------------------
 1 experimental option(s) unlocked:
 - '-H:CompilerBackend' (origin(s): command line)
------------------------------------------------------------------------------------------------------------------------
Build resources:
 - 26.49GB of memory (10.6% of 250.20GB system memory, determined at start)
 - 32 thread(s) (40.0% of 80 available processor(s), determined at start)
[2/8] Performing analysis...  [******]                                                                  (55.9s @ 0.39GB)
    3,272 reachable types   (73.5% of    4,449 total)
    3,809 reachable fields  (50.2% of    7,590 total)
   15,495 reachable methods (45.5% of   34,061 total)
    1,035 types,    87 fields, and   673 methods registered for reflection
       57 types,    55 fields, and    52 methods registered for JNI access
        5 native libraries: dl, m, pthread, rt, z
[3/8] Building universe...                                                                              (14.1s @ 0.90GB)
[4/8] Parsing methods...      [*]                                                                        (1.2s @ 0.53GB)
[5/8] Inlining methods...     [***]                                                                      (1.0s @ 0.94GB)
[6/8] Compiling methods...    [******]                                                                  (42.1s @ 0.50GB)
[7/8] Layouting methods...    [*****]                                                                   (24.4s @ 0.55GB)

31 fatal errors detected:
Fatal error: org.graalvm.compiler.debug.GraalError: LLVM compilation failed for batch 24 (f6840-f7125). Use -H:LLVMMaxFunctionsPerBatch=1 to compile each method individually. (/tmp/SVM-2481504409853990114/llvm/b24o.bc): 1
Command: llc -relocation-model=pic --trap-unreachable -march=aarch64 --frame-pointer=all --aarch64-frame-record-on-top -O2 -filetype=obj -o b24.o b24o.bc
    at org.graalvm.nativeimage.llvm/com.oracle.svm.core.graal.llvm.LLVMToolchainUtils.llvmCompile(LLVMToolchainUtils.java:108)

Full log: log.txt

Bundle: Main.nib.zip

My LLVM toolchain is a stock one from a package:

$ clang --version
clang version 16.0.6 (Red Hat 16.0.6-1.el9)
Target: aarch64-redhat-linux-gnu
Thread model: posix
InstalledDir: /usr/bin

My system:

$ uname -a
Linux mandrel2 5.14.0-319.el9.aarch64 #1 SMP PREEMPT_DYNAMIC Thu May 25 20:42:07 UTC 2023 aarch64 aarch64 aarch64 GNU/Linux
$ cat /etc/redhat-release 
CentOS Stream release 9
$ getconf PAGESIZE
4096

Why would I need that, there is an AARCH64 target...

My motivation is to explore the possibility of swapping the vanilla llvm clang with afl-clang, from AFL to speed up experiments with build time instrumentation as runtime instrumentation (Frida, QEMU) is abysmally slow in my scenario.

[Karm]

I was advised to patch my own llvm build with https://github.com/oracle/graal/tree/master/sdk/llvm-patches/native-image I'm gonna try.

[Karm]

It works for me with the patched LLVM:

$ strings Main.output/default/Main | grep CCompiler
com.oracle.svm.core.VM.Target.CCompiler=clang|llvm|aarch64|16.0.1

[ThomasTNO]

@Karm, did you manage to get it operational with afl-clang?

[Karm]

Hello @ThomasTNO,

I did not :disappointed:

I have two avenues I am exploring with GraalVM native-image built binaries:

• on my amd64 workstation, I am using Frida backend to AFL
• on my aarch64 workstation, I have QEMU user space emulation going on as an AFL backend

The target is a rather complex Quarkus app, so it's too slow to be meaningful yet, I hit like ~10 attempts per second per core :) Zzzzz

I am yet to narrow down the persistent mode and I think I will also chop it up into smaller, isolated, much faster executables.

afl-clang

The instrumentation AFL Plus Plus does here is way too early in the pipeline, before lowering to IR. GraalVM native-image actually doesn't invoke clang command itself at all. It prepares LLVM bitcode and it calls lli and opt to chew the bitcode, not clang to emit it. I am yet to see if I could either implement what AFL does in GraalVM or figure something else out...

I will prepare some post about it when I have it narrowed down.

Are you interested in fuzzing native-image made executables? My main aim are situations where plain JVM with HotSpot would recover, e.g. with an exception, but native-image executable process gets killed, e.g. taking a whole Quarkus app down.