search

jeffque / testing-jvmci

Running Peter Ackerman (trampoline version) to test JVMCI speed ups
2 stars 0 forks source link

Add (1) GraalCE Native Image, (2) Oracle Graal Jit and Native Image for comparison purposes and (3) Consider JMH #1

Open lobaorn opened 1 month ago

lobaorn commented 1 month ago

Hi @jeffque I was thinking that we should add the Native Image and as well both JIT and Native Image from Oracle GraalVM, to properly compare "all" options. And perhaps integrate it better with JMH.

I am unsure if the people from GraalVM has some foundation on how to do these types of tests/benchmarks.

Could you give an insight here @chumer @alina-yur ?

chumer commented 1 month ago

JMH should work on GraalVM CE and Oracle GraalVM on HotSpot or Native Image using the same configuration. Maybe there are some hickups with building the native-image with JMH but i can give you some pointers if it does not work out of the box.

Here is a sample JMH benchmark from Graal.js: https://github.com/oracle/graaljs/blob/fad9af323bbf014168bda8f1aae3c96b08c7d33e/graal-js/src/com.oracle.truffle.js.jmh/src/com/oracle/truffle/js/jmh/JMHArrayInteropBenchmark.java#L60

Also it can make sense to context.enter() in a setup routine to remove the overhead of entering a context from the benchmark.

Note that we do most of our benchmarking using our standalone builds and with their respective runtime executables (Node, python, ...) and not using Java embeddings. We also use our own thing called the Polybench internally, which might be a bit hard to use atm without our infrastructure, but could work as a starting point for your own thing: https://github.com/oracle/graal/blob/ce344dd45cc746790b06bc572086a922b2ae328c/vm/src/org.graalvm.polybench/src/org/graalvm/polybench/PolyBenchLauncher.java#L53