gunnarmorling / 1brc

1️⃣🐝🏎️ The One Billion Row Challenge -- A fun exploration of how quickly 1B rows from a text file can be aggregated with Java
https://www.morling.dev/blog/one-billion-row-challenge/
Apache License 2.0
6.14k stars 1.85k forks source link

Going for the 1st place #681

Closed gonix closed 8 months ago

gonix commented 8 months ago

I saw child process spawning trick in other solutions and figured I could improve on it a little bit. Obviously there's still some room for improvement, but I believe I've already achieved O(0.1) execution time, so why try harder?

Check List:


P.S. this is just for laughs ;-)

gonix commented 8 months ago

For giggles the record:

$ ./evaluate.sh gonix1st
WARNING SMT is enabled
+ sdk use java 21.0.1-open

Using java version 21.0.1-open in this shell.
+ java --version
openjdk 21.0.1 2023-10-17
OpenJDK Runtime Environment (build 21.0.1+12-29)
OpenJDK 64-Bit Server VM (build 21.0.1+12-29, mixed mode, sharing)
+ ./mvnw --quiet clean verify
+ rm -f measurements.txt
+ ln -s measurements_1B.txt measurements.txt

+ sdk use java 21.0.1-open

Using java version 21.0.1-open in this shell.
+ timeout -v 300 ./test.sh gonix1st
Validating calculate_average_gonix1st.sh -- src/test/resources/samples/measurements-10000-unique-keys.txt
Validating calculate_average_gonix1st.sh -- src/test/resources/samples/measurements-10.txt
Validating calculate_average_gonix1st.sh -- src/test/resources/samples/measurements-1.txt
Validating calculate_average_gonix1st.sh -- src/test/resources/samples/measurements-20.txt
Validating calculate_average_gonix1st.sh -- src/test/resources/samples/measurements-2.txt
Validating calculate_average_gonix1st.sh -- src/test/resources/samples/measurements-3.txt
Validating calculate_average_gonix1st.sh -- src/test/resources/samples/measurements-boundaries.txt
Validating calculate_average_gonix1st.sh -- src/test/resources/samples/measurements-complex-utf8.txt
Validating calculate_average_gonix1st.sh -- src/test/resources/samples/measurements-dot.txt
Validating calculate_average_gonix1st.sh -- src/test/resources/samples/measurements-rounding.txt
Validating calculate_average_gonix1st.sh -- src/test/resources/samples/measurements-shortest.txt
Validating calculate_average_gonix1st.sh -- src/test/resources/samples/measurements-short.txt

+ timeout -v 300 ./test.sh gonix1st measurements_1B.txt
Validating calculate_average_gonix1st.sh -- measurements_1B.txt

+ rm -f measurements.txt
+ ln -s measurements_1B.txt measurements.txt
Benchmark 1: timeout -v 300 ./calculate_average_gonix1st.sh 2>&1
  Time (mean ± σ):     110.3 ms ±   4.3 ms    [User: 1.4 ms, System: 0.1 ms]
  Range (min … max):   105.7 ms … 115.7 ms    5 runs

Summary
  gonix1st: trimmed mean 0.11002448828, raw times 0.11064476328,0.10644592628,0.11298277528,0.11566057628000001,0.10565949328

Leaderboard

| # | Result (m:s.ms) | Implementation     | JDK | Submitter     | Notes     |
|---|-----------------|--------------------|-----|---------------|-----------|
|   | 00:00.110 | [link](https://github.com/gunnarmorling/1brc/blob/main/src/main/java/dev/morling/onebrc/CalculateAverage_gonix1st.java)| 21.0.1-open | [gonix1st](https://github.com/gonix1st) |  |

Raw results saved to file(s):
  gonix1st-20240131195839.out
gunnarmorling commented 8 months ago

Haha. It's a bit too fast, so I'll have to reject it, unfortunately. Very creative though ;)