meilisearch / grenad

Tools to sort, merge, write, and read immutable key-value pairs :tomato:
https://docs.rs/grenad
MIT License
25 stars 3 forks source link

Speed-up the Sorter #40

Open Kerollmops opened 1 year ago

Kerollmops commented 1 year ago

The Sorter is doing a lot of operations to ensure the values are merged in the correct order.

I thought about a simple way to speed the sort operation a little bit: store the first byte(s) of the key in the pointer of the EntryBound itself, this way we will reduce the data fetching from the other side of the in-memory buffer. We could use an equivalent of the C bitfields.

We can then conditionally fetch the key at the other side of the buffer only if both first bytes of the two compared keys are equal. We will need to use the <[EntryBound]>::sort_by variant and no more the <EntryBound>::sort_by_key one.