gunnarmorling / 1brc

1️⃣🐝🏎️ The One Billion Row Challenge -- A fun exploration of how quickly 1B rows from a text file can be aggregated with Java
https://www.morling.dev/blog/one-billion-row-challenge/
Apache License 2.0
6k stars 1.8k forks source link

Exit earlier from loop when a new Result is created #668

Closed tivrfoa closed 6 months ago

tivrfoa commented 7 months ago
 *   3) Make a cache of long[] name to String, to avoid `ByteBuffer.allocate`
 * and creating new UTF-8 strings. I didn't profile, so it's just a guess
 * that this map will be a bit faster. Although it's outside the main loop, so
 * not a big difference ...;
 *   4) Exit earlier from loop if a new entry was created.

Check List:

gunnarmorling commented 6 months ago

A bit slower now:

Benchmark 1: timeout -v 300 ./calculate_average_tivrfoa.sh 2>&1
  Time (mean ± σ):      2.355 s ±  0.022 s    [User: 0.002 s, System: 0.004 s]
  Range (min … max):    2.320 s …  2.375 s    5 runs

Summary
  tivrfoa: trimmed mean 2.3596067892600003, raw times 2.3198986672600004,2.3745432172600003,2.36888179626,2.3620232222600004,2.3479153492600005

Leaderboard

| # | Result (m:s.ms) | Implementation     | JDK | Submitter     | Notes     |
|---|-----------------|--------------------|-----|---------------|-----------|
|   | 00:02.359 | [link](https://github.com/gunnarmorling/1brc/blob/main/src/main/java/dev/morling/onebrc/CalculateAverage_tivrfoa.java)| 21.0.2-graal | [tivrfoa](https://github.com/tivrfoa) | GraalVM native binary, uses Unsafe |
tivrfoa commented 6 months ago

A bit slower now:

Thank you @gunnarmorling !

I made some changes. Could you please run it again?

gunnarmorling commented 6 months ago

Still slower:

Benchmark 1: timeout -v 300 ./calculate_average_tivrfoa.sh 2>&1
  Time (mean ± σ):      2.287 s ±  0.007 s    [User: 0.001 s, System: 0.005 s]
  Range (min … max):    2.281 s …  2.298 s    5 runs

Summary
  tivrfoa: trimmed mean 2.284516698433333, raw times 2.2978994541,2.2853571351,2.2840650791,2.2841278811,2.2813756321

Leaderboard

| # | Result (m:s.ms) | Implementation     | JDK | Submitter     | Notes     |
|---|-----------------|--------------------|-----|---------------|-----------|
|   | 00:02.284 | [link](https://github.com/gunnarmorling/1brc/blob/main/src/main/java/dev/morling/onebrc/CalculateAverage_tivrfoa.java)| 21.0.2-graal | [tivrfoa](https://github.com/tivrfoa) | GraalVM native binary, uses Unsafe |
tivrfoa commented 6 months ago

Hi @gunnarmorling

The new code will be slower, but you can merge it this way, no problem. If you could just run the code on master for the 10k before, just curious to see the previous time.

Thanks a lot for 1brc!

I think #1brc will be the best thing for Java in 2024, and we're just in January!!! =D

gunnarmorling commented 6 months ago

00:02.995 now for the standard key set. Amongst the top for 10K (I'll update that leaderboard later this week):

# Result (m:s.ms) Implementation JDK Submitter Notes
00:04.570 link 21.0.2-graal tivrfoa GraalVM native binary, uses Unsafe

I think #1brc will be the best thing for Java in 2024, and we're just in January!!! =D

Thank you so much, I really appreciate it! Thanks for participating in 1BRC and congrats on creating one of the top entries!

gunnarmorling commented 6 months ago

Hey @tivrfoa!

Congrats again on being in the Top 20 of the One Billion Row Challenge!

To celebrate this amazing achievement, I would like to send you a 1BRC t-shirt and coffee mug. To claim your prize, fill out this form by Feb 18. After submitting the form, please provide a comment with the random value you've specified in the form, so that I know it is you who submitted it.

All data entered will solely be used in relation to processing this shipment. Shipments can be sent to any country listed here or here (I'll use whichever one is cheaper for me to ship to your location). A big thank you to Decodable for sponsoring these prizes!

Thanks a lot for participating in 1BRC,

--Gunnar

tivrfoa commented 6 months ago

Hey @tivrfoa!

Congrats again on being in the Top 20 of the One Billion Row Challenge!

To celebrate this amazing achievement, I would like to send you a 1BRC t-shirt and coffee mug. To claim your prize, fill out this form by Feb 18. After submitting the form, please provide a comment with the random value you've specified in the form, so that I know it is you who submitted it.

All data entered will solely be used in relation to processing this shipment. Shipments can be sent to any country listed here or here (I'll use whichever one is cheaper for me to ship to your location). A big thank you to Decodable for sponsoring these prizes!

Thanks a lot for participating in 1BRC,

--Gunnar

32228 24253

Thank you! :smile: