search

Tessil / hat-trie

C++ implementation of a fast and memory efficient HAT-trie
MIT License
792 stars 114 forks source link

hat-trie behaves wrong when burst-threshold is set to 65536 or larger #54

Open liuxiaochuang opened 8 months ago

liuxiaochuang commented 8 months ago

When the burst-threshold is set to smaller than 65536, the hat-trie behaves normally, and memory usage is 32%. However, when set to 65536 or larger, it causes my process to coredump, with memory usage only at 24%. Why is that?

Tessil commented 2 months ago

Thanks for the report.

There's effectively a limit of 65535 due to the IndexSizeT of uint16 used for the array_hash of the htrie_hash.

I added a check in commit 611bdccbe40f4bfb7f75d849c5bec6a003695db8. Previously, an exception should have been thrown if we were inserting more than 65535 elements. Did you get a segfault instead?