search

alecthomas / mph

Minimal Perfect Hashing for Go
BSD 3-Clause "New" or "Revised" License
177 stars 23 forks source link

Reduce the number of pointers created in Mmap() #16

Closed Jille closed 9 months ago

Jille commented 9 months ago

Just use ints pointing into the large byte slice.

Mmap() creates two []bytes per key value pair. This creates significant work for the garbage collector which has to scan through all of these in each mark phase. For my 2.7M keys, it adds 5.4M pointers to be scanned. By just using integers for offsets the garbage collector can ignore it completely.

This does trade off some more work for the Builder which has to append all the keys+values into a single byte slice.

Jille commented 9 months ago

@SnoozeThis https://github.com/alecthomas/mph/pull/15

SnoozeThis commented 9 months ago

(https://snoozeth.is/tHDCudkwLCw) I will wait until https://github.com/alecthomas/mph/pull/15 is merged and then add a comment.

alecthomas commented 9 months ago

Can you consolidate these into one PR please? It just creates more work for me if there are N different PRs.

Jille commented 9 months ago

Sure! Most projects prefer them all split up, so I went that route initially :)

https://github.com/alecthomas/mph/pull/17

SnoozeThis commented 9 months ago

An error occurred while snoozing: Pull request was closed without being merged