gunnarmorling / 1brc

1️⃣🐝🏎️ The One Billion Row Challenge -- A fun exploration of how quickly 1B rows from a text file can be aggregated with Java
https://www.morling.dev/blog/one-billion-row-challenge/
Apache License 2.0
6.08k stars 1.83k forks source link

More small tweaks, perf from 77.. to 75.., another 2-3%? #577

Closed royvanrijn closed 7 months ago

royvanrijn commented 7 months ago

Added some options in the prepare file, makes a small difference, also tweaked the code to eliminate even more cycles.

Check List:

gunnarmorling commented 7 months ago

OOMs now on the eval machine:

+ timeout -v 300 ./test.sh royvanrijn

Using java version 21.0.2-graal in this shell.
Validating calculate_average_royvanrijn.sh -- src/test/resources/samples/measurements-10000-unique-keys.txt
Picking up existing native image 'target/CalculateAverage_royvanrijn_image', delete the file to select JVM mode.
Exception in thread "Thread-27" java.lang.OutOfMemoryError: Garbage-collected heap size exceeded. Consider increasing the maximum Java heap size, for example with '-Xmx'.
Exception in thread "main" java.lang.OutOfMemoryError: Garbage-collected heap size exceeded. Consider increasing the maximum Java heap size, for example with '-Xmx'.
Exception in thread "Thread-23" java.lang.OutOfMemoryError: Garbage-collected heap size exceeded. Consider increasing the maximum Java heap size, for example with '-Xmx'.
Exception in thread "Thread-28" java.lang.OutOfMemoryError: Garbage-collected heap size exceeded. Consider increasing the maximum Java heap size, for example with '-Xmx'.
Fatal error: java.lang.OutOfMemoryError: Garbage-collected heap size exceeded. Consider increasing the maximum Java heap size, for example with '-Xmx'.
royvanrijn commented 7 months ago

Ah strange, “worked on my machine”.

I’ll update the PR tomorrow with more lenient settings and some small tweaks.