Performance comparison

[KengoTODA]

export JAVA_HOME=path/to/graalvm-21.1.0
export JAVA8_HOME=path/to/jdk8
./gradlew assemble nativeImage
pushd build/distributions && unzip *.zip && popd

hyperfine --ignore-failure \
  './build/executable/spotbugs -Djava.home=$JAVA8_HOME --aux $JAVA8_HOME/jre/lib/rt.jar --xml=build/spotbugs.xml build/spotbugs/spotbugs-4.4.3-SNAPSHOT/lib/spotbugs.jar' \
  './build/distributions/spotbugs-cli-1.0.0-SNAPSHOT/bin/spotbugs-cli --xml=build/spotbugs.xml build/spotbugs/spotbugs-4.4.3-SNAPSHOT/lib/spotbugs.jar'

[KengoTODA]

Ubuntu 18.04

native image is 1.12 ± 0.07 times faster than java

Benchmark #1: ./build/executable/spotbugs -Djava.home=$JAVA8_HOME --aux $JAVA8_HOME/jre/lib/rt.jar --xml=build/spotbugs.xml build/spotbugs/spotbugs-4.4.3-SNAPSHOT/lib/spotbugs.jar
  Time (mean ± σ):     74.902 s ±  2.375 s    [User: 70.418 s, System: 4.362 s]
  Range (min … max):   72.189 s … 78.325 s    10 runs

  Warning: Ignoring non-zero exit code.

Benchmark #2: ./build/distributions/spotbugs-cli-1.0.0-SNAPSHOT/bin/spotbugs-cli --xml=build/spotbugs.xml build/spotbugs/spotbugs-4.4.3-SNAPSHOT/lib/spotbugs.jar
  Time (mean ± σ):     84.228 s ±  4.340 s    [User: 197.464 s, System: 7.295 s]
  Range (min … max):   80.306 s … 91.507 s    10 runs

  Warning: Ignoring non-zero exit code.

Summary
  './build/executable/spotbugs -Djava.home=$JAVA8_HOME --aux $JAVA8_HOME/jre/lib/rt.jar --xml=build/spotbugs.xml build/spotbugs/spotbugs-4.4.3-SNAPSHOT/lib/spotbugs.jar' ran
    1.12 ± 0.07 times faster than './build/distributions/spotbugs-cli-1.0.0-SNAPSHOT/bin/spotbugs-cli --xml=build/spotbugs.xml build/spotbugs/spotbugs-4.4.3-SNAPSHOT/lib/spotbugs.jar'

[KengoTODA]

macOS 12.0.1

java is 1.15 ± 0.02 times faster than native image

Benchmark #1: ./build/executable/spotbugs -Djava.home=$JAVA8_HOME --aux $JAVA8_HOME/jre/lib/rt.jar --xml=build/spotbugs.xml build/spotbugs/spotbugs-4.4.3-SNAPSHOT/lib/spotbugs.jar
  Time (mean ± σ):     62.989 s ±  0.772 s    [User: 58.404 s, System: 4.460 s]
  Range (min … max):   62.172 s … 64.948 s    10 runs

  Warning: Ignoring non-zero exit code.

Benchmark #2: ./build/distributions/spotbugs-cli-1.0.0-SNAPSHOT/bin/spotbugs-cli --xml=build/spotbugs.xml build/spotbugs/spotbugs-4.4.3-SNAPSHOT/lib/spotbugs.jar
  Time (mean ± σ):     54.873 s ±  0.962 s    [User: 133.125 s, System: 2.967 s]
  Range (min … max):   53.831 s … 56.626 s    10 runs

  Warning: Ignoring non-zero exit code.

Summary
  './build/distributions/spotbugs-cli-1.0.0-SNAPSHOT/bin/spotbugs-cli --xml=build/spotbugs.xml build/spotbugs/spotbugs-4.4.3-SNAPSHOT/lib/spotbugs.jar' ran
    1.15 ± 0.02 times faster than './build/executable/spotbugs -Djava.home=$JAVA8_HOME --aux $JAVA8_HOME/jre/lib/rt.jar --xml=build/spotbugs.xml build/spotbugs/spotbugs-4.4.3-SNAPSHOT/lib/spotbugs.jar'

[KengoTODA]

java is 1.07 ± 0.04 times faster than native image

With GraalVM 21.3.0 on macOS 12.0.1

hyperfine --ignore-failure \
  './build/executable/spotbugs -Djava.home=$JAVA8_HOME --aux $JAVA8_HOME/jre/lib/rt.jar --xml=build/spotbugs.xml build/spotbugs/spotbugs-4.5.1-SNAPSHOT/lib/spotbugs.jar' \
  './build/distributions/spotbugs-cli-1.0.0-SNAPSHOT/bin/spotbugs-cli --xml=build/spotbugs.xml build/spotbugs/spotbugs-4.5.1-SNAPSHOT/lib/spotbugs.jar'

Benchmark #1: ./build/executable/spotbugs -Djava.home=$JAVA8_HOME --aux $JAVA8_HOME/jre/lib/rt.jar --xml=build/spotbugs.xml build/spotbugs/spotbugs-4.5.1-SNAPSHOT/lib/spotbugs.jar
  Time (mean ± σ):     63.636 s ±  1.610 s    [User: 60.303 s, System: 3.051 s]
  Range (min … max):   62.524 s … 67.854 s    10 runs

  Warning: Ignoring non-zero exit code.

Benchmark #2: ./build/distributions/spotbugs-cli-1.0.0-SNAPSHOT/bin/spotbugs-cli --xml=build/spotbugs.xml build/spotbugs/spotbugs-4.5.1-SNAPSHOT/lib/spotbugs.jar
  Time (mean ± σ):     59.338 s ±  1.953 s    [User: 141.918 s, System: 3.233 s]
  Range (min … max):   56.715 s … 61.363 s    10 runs

  Warning: Ignoring non-zero exit code.

Summary
  './build/distributions/spotbugs-cli-1.0.0-SNAPSHOT/bin/spotbugs-cli --xml=build/spotbugs.xml build/spotbugs/spotbugs-4.5.1-SNAPSHOT/lib/spotbugs.jar' ran
    1.07 ± 0.04 times faster than './build/executable/spotbugs -Djava.home=$JAVA8_HOME --aux $JAVA8_HOME/jre/lib/rt.jar --xml=build/spotbugs.xml build/spotbugs/spotbugs-4.5.1-SNAPSHOT/lib/spotbugs.jar'