Clean up, fine tuning, credit section for thomaswue

thomaswue commented 7 months ago

section and additional comments.

Check List:

[x] You have run ./mvnw verify and the project builds successfully
[x] Tests pass (./test.sh <username> shows no differences between expected and actual outputs)
[x] All formatting changes by the build are committed
[x] Your launch script is named calculate_average_<username>.sh (make sure to match casing of your GH user name) and is executable
[x] Output matches that of calculate_average_baseline.sh
[x] For new entries, or after substantial changes: When implementing custom hash structures, please point to where you deal with hash collisions (line number)
Execution time: 0.40
Execution time of reference implementation: 120.37

This is primarily for clean up and adding a credits section (including now also @artsiomkorzun). Furthermore, it removes some of the experimental native image options and exchanges for supported flags. This does run ~3% better on my machine, but unclear if this translates to anything on the evaluation machine (hopefully no regression ;)).

gunnarmorling commented 7 months ago

Getting this build failure now, I suppose due to --static?

> java.lang.RuntimeException: There was an error linking the native image: Linker command exited with 1

Based on the linker command output, possible reasons for this include:
1. It appears as though libc.a is missing. Please install it.

Linker command executed:
/usr/bin/gcc -z noexecstack -Wl,--gc-sections -Wl,--version-script,/tmp/SVM-7162692315681591361/exported_symbols.list -Wl,-x -o /home/build/projects/eval/1brc/target/CalculateAverage_thomaswue_image CalculateAverage_thomaswue_image.o /home/build/.sdkman/candidates/java/21.0.2-graal/lib/svm/clibraries/linux-amd64/liblibchelper.a /home/build/.sdkman/candidates/java/21.0.2-graal/lib/static/linux-amd64/glibc/libnet.a /home/build/.sdkman/candidates/java/21.0.2-graal/lib/static/linux-amd64/glibc/libnio.a /home/build/.sdkman/candidates/java/21.0.2-graal/lib/static/linux-amd64/glibc/libjava.a /home/build/.sdkman/candidates/java/21.0.2-graal/lib/static/linux-amd64/glibc/libzip.a /home/build/.sdkman/candidates/java/21.0.2-graal/lib/svm/clibraries/linux-amd64/libjvm.a -static -v -L/tmp/SVM-7162692315681591361 -L/home/build/.sdkman/candidates/java/21.0.2-graal/lib/static/linux-amd64/glibc -L/home/build/.sdkman/candidates/java/21.0.2-graal/lib/svm/clibraries/linux-amd64 -lz -ldl -lpthread -lrt

Linker command output:
Using built-in specs.
COLLECT_GCC=/usr/bin/gcc
COLLECT_LTO_WRAPPER=/usr/libexec/gcc/x86_64-redhat-linux/11/lto-wrapper
OFFLOAD_TARGET_NAMES=nvptx-none
OFFLOAD_TARGET_DEFAULT=1
Target: x86_64-redhat-linux
Configured with: ../configure --enable-bootstrap --enable-host-pie --enable-host-bind-now --enable-languages=c,c++,fortran,lto --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info --with-bugurl=http://bugzilla.redhat.com/bugzilla --enable-shared --enable-threads=posix --enable-checking=release --with-system-zlib --enable-__cxa_atexit --disable-libunwind-exceptions --enable-gnu-unique-object --enable-linker-build-id --with-gcc-major-version-only --enable-plugin --enable-initfini-array --without-isl --enable-multilib --with-linker-hash-style=gnu --enable-offload-targets=nvptx-none --without-cuda-driver --enable-gnu-indirect-function --enable-cet --with-tune=generic --with-arch_64=x86-64-v2 --with-arch_32=x86-64 --build=x86_64-redhat-linux --with-build-config=bootstrap-lto --enable-link-serialization=1
Thread model: posix
Supported LTO compression algorithms: zlib zstd
gcc version 11.4.1 20231218 (Red Hat 11.4.1-3) (GCC)
COMPILER_PATH=/usr/libexec/gcc/x86_64-redhat-linux/11/:/usr/libexec/gcc/x86_64-redhat-linux/11/:/usr/libexec/gcc/x86_64-redhat-linux/:/usr/lib/gcc/x86_64-redhat-linux/11/:/usr/lib/gcc/x86_64-redhat-linux/
LIBRARY_PATH=/usr/lib/gcc/x86_64-redhat-linux/11/:/usr/lib/gcc/x86_64-redhat-linux/11/../../../../lib64/:/lib/../lib64/:/usr/lib/../lib64/:/usr/lib/gcc/x86_64-redhat-linux/11/../../../:/lib/:/usr/lib/
COLLECT_GCC_OPTIONS='-z' 'noexecstack' '-o' '/home/build/projects/eval/1brc/target/CalculateAverage_thomaswue_image' '-static' '-v' '-L/tmp/SVM-7162692315681591361' '-L/home/build/.sdkman/candidates/java/21.0.2-graal/lib/static/linux-amd64/glibc' '-L/home/build/.sdkman/candidates/java/21.0.2-graal/lib/svm/clibraries/linux-amd64' '-mtune=generic' '-march=x86-64-v2' '-dumpdir' '/home/build/projects/eval/1brc/target/CalculateAverage_thomaswue_image.'
 /usr/libexec/gcc/x86_64-redhat-linux/11/collect2 -plugin /usr/libexec/gcc/x86_64-redhat-linux/11/liblto_plugin.so -plugin-opt=/usr/libexec/gcc/x86_64-redhat-linux/11/lto-wrapper -plugin-opt=-fresolution=/tmp/ccd7uFHH.res -plugin-opt=-pass-through=-lgcc -plugin-opt=-pass-through=-lgcc_eh -plugin-opt=-pass-through=-lc --build-id --no-add-needed --hash-style=gnu -m elf_x86_64 -static -o /home/build/projects/eval/1brc/target/CalculateAverage_thomaswue_image -z noexecstack /usr/lib/gcc/x86_64-redhat-linux/11/../../../../lib64/crt1.o /usr/lib/gcc/x86_64-redhat-linux/11/../../../../lib64/crti.o /usr/lib/gcc/x86_64-redhat-linux/11/crtbeginT.o -L/tmp/SVM-7162692315681591361 -L/home/build/.sdkman/candidates/java/21.0.2-graal/lib/static/linux-amd64/glibc -L/home/build/.sdkman/candidates/java/21.0.2-graal/lib/svm/clibraries/linux-amd64 -L/usr/lib/gcc/x86_64-redhat-linux/11 -L/usr/lib/gcc/x86_64-redhat-linux/11/../../../../lib64 -L/lib/../lib64 -L/usr/lib/../lib64 -L/usr/lib/gcc/x86_64-redhat-linux/11/../../.. --gc-sections --version-script /tmp/SVM-7162692315681591361/exported_symbols.list -x CalculateAverage_thomaswue_image.o /home/build/.sdkman/candidates/java/21.0.2-graal/lib/svm/clibraries/linux-amd64/liblibchelper.a /home/build/.sdkman/candidates/java/21.0.2-graal/lib/static/linux-amd64/glibc/libnet.a /home/build/.sdkman/candidates/java/21.0.2-graal/lib/static/linux-amd64/glibc/libnio.a /home/build/.sdkman/candidates/java/21.0.2-graal/lib/static/linux-amd64/glibc/libjava.a /home/build/.sdkman/candidates/java/21.0.2-graal/lib/static/linux-amd64/glibc/libzip.a /home/build/.sdkman/candidates/java/21.0.2-graal/lib/svm/clibraries/linux-amd64/libjvm.a -lz -ldl -lpthread -lrt --start-group -lgcc -lgcc_eh -lc --end-group /usr/lib/gcc/x86_64-redhat-linux/11/crtend.o /usr/lib/gcc/x86_64-redhat-linux/11/../../../../lib64/crtn.o
/usr/bin/ld: cannot find -lz
/usr/bin/ld: cannot find -lc
collect2: error: ld returned 1 exit status

Any idea which packages I need to install for that? Does it make a substantial difference?

thomaswue commented 7 months ago

This is just for a few milliseconds. The static linking makes the native image binary start up slightly faster. I removed the option. Here is some more info in case anybody wants to experiment with this: https://www.graalvm.org/latest/reference-manual/native-image/guides/build-static-executables/

gunnarmorling commented 7 months ago

Thanks, that makes my life a bit simpler :)

And still coming in a tad faster than what's currently on main:

Benchmark 1: timeout -v 300 ./calculate_average_thomaswue.sh 2>&1
  Time (mean ± σ):      1.879 s ±  0.005 s    [User: 0.002 s, System: 0.003 s]
  Range (min … max):    1.871 s …  1.887 s    10 runs

Summary
  thomaswue: trimmed mean 1.8787649873399999, raw times 1.87129162834,1.87679895034,1.88153617834,1.87835341434,1.88549133934,1.8872258233399999,1.87726399134,1.87159845334,1.8792556653399999,1.87982190634

Leaderboard

| # | Result (m:s.ms) | Implementation     | JDK | Submitter     | Notes     |
|---|-----------------|--------------------|-----|---------------|-----------|
|   | 00:01.878 | [link](https://github.com/gunnarmorling/1brc/blob/main/src/main/java/dev/morling/onebrc/CalculateAverage_thomaswue.java)| 21.0.2-graal | [Thomas Wuerthinger](https://github.com/thomaswue) | GraalVM native binary, uses Unsafe |

thomaswue commented 7 months ago

Cool, thanks for merging! Every millisecond counts at this point, so I take those 15 milliseconds faster ;-).

gunnarmorling / 1brc

Clean up, fine tuning, credit section for thomaswue #646

Check List: