gunnarmorling / 1brc

1️⃣🐝🏎️ The One Billion Row Challenge -- A fun exploration of how quickly 1B rows from a text file can be aggregated with Java
https://www.morling.dev/blog/one-billion-row-challenge/
Apache License 2.0
6k stars 1.8k forks source link

jerrinot's improvement - everyone is hunting ILP? let's do less! #652

Closed jerrinot closed 6 months ago

jerrinot commented 7 months ago

it turns out doing 3 things at once was too much. perf annotate showed spilling.

Check List:

thanks to @franz1981 for the idea to explore register spilling and to @thomaswue for tips on how to do it!

gunnarmorling commented 6 months ago

@jerrinot, where does it handle hash collisions actually?

jerrinot commented 6 months ago

closing in favour of https://github.com/gunnarmorling/1brc/pull/690

jerrinot commented 6 months ago

@gunnarmorling: it uses 2 maps:

  1. slow map, which can store long names. the name is checked here: https://github.com/jerrinot/1brc/blob/f76e8d27c706cc785a79c6f598202a0d9d5e2566/src/main/java/dev/morling/onebrc/CalculateAverage_jerrinot.java#L571-L575
  2. fast-map. which can store only names up to 15 bytes. and it's returning only when both words are matching: https://github.com/jerrinot/1brc/blob/f76e8d27c706cc785a79c6f598202a0d9d5e2566/src/main/java/dev/morling/onebrc/CalculateAverage_jerrinot.java#L519-L521
gunnarmorling commented 6 months ago

Got it, thx!

Message ID: @.***>