shipilev: Amendments to version 4 - Githubissues

gunnarmorling / 1brc

1️⃣🐝🏎️ The One Billion Row Challenge -- A fun exploration of how quickly 1B rows from a text file can be aggregated with Java

https://www.morling.dev/blog/one-billion-row-challenge/

Apache License 2.0

6k stars 1.8k forks source link

shipilev: Amendments to version 4 #627

Closed shipilev closed 7 months ago

shipilev commented 7 months ago

Check List:

[x] Tests pass (./test.sh <username> shows no differences between expected and actual outputs)
[x] All formatting changes by the build are committed
[x] Your launch script is named calculate_average_<username>.sh (make sure to match casing of your GH user name) and is executable
[x] Output matches that of calculate_average_baseline.sh
[x] ~For new entries, or after substantial changes~ Not a new entry.

A few simple touchups after running with 8 cores:

Default FJP pool is one thread short when execution is capped at 8 cores;
No need for some JVM options in the constrained run;
Tiny locality improvements;

On standard dataset:

# Old impl:
Benchmark 1: ./calculate_average_shipilev.sh
  Time (mean ± σ):      3.819 s ±  0.091 s    [User: 25.632 s, System: 0.982 s]
  Range (min … max):    3.677 s …  3.990 s    10 runs

# New impl:
Benchmark 1: ./calculate_average_shipilev.sh
  Time (mean ± σ):      3.252 s ±  0.086 s    [User: 24.463 s, System: 0.906 s]
  Range (min … max):    3.196 s …  3.489 s    10 runs

So it should be about 15% faster.

gunnarmorling commented 7 months ago

00:04.884 now. As for your question on the original PR, the official eval is on 8 cores indeed, but I am doing an occasional side eval on all the cores, with SMT, for funsies.