meilisearch / grenad

Tools to sort, merge, write, and read immutable key-value pairs :tomato:
https://docs.rs/grenad
MIT License
25 stars 3 forks source link

Correctly write a chunk when the number of entries reach max-memory #2

Closed Kerollmops closed 3 years ago

Kerollmops commented 3 years ago

This PR correctly triggers writing the in-memory buffer to disk when the number of entries reached the max-memory limit, the previous code when checking that the capacity of the entries buffer reached the threshold, unfortunately, the capacity was never shrunk and therefore the system was always triggered, at ever insert.

Note that the entries buffer capacity represents the real amount of memory consumed by the sorter, using the length of it is kind of wrong, but the amount of memory represented by the length doesn't outfit the max-memory by far compared to the capacity.

This PR closes #1 that is fixing an imaginary bug.