laysakura / trie-rs

Memory efficient trie (prefix tree) library based on LOUDS
https://crates.io/crates/trie-rs
Apache License 2.0
90 stars 10 forks source link

Reduce memory usage #31

Open shanecelis opened 4 months ago

shanecelis commented 4 months ago

Now that we have mem_dbg present, we can determine the memory usage. And presently the trie is larger than the dictionary in memory, which surprised me.

$ cargo test --features mem_dbg memsize -- --nocapture
running 1 test
Reading dictionary file from: /Users/shane/Projects/trie-rs/benches/edict.furigana
Read 185536 words, 3531819 bytes.
Trie size 4174393
Uncompressed size 7984707
test trie::trie_impl::search_tests::memsize ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 114 filtered out; finished in 7.17s

I fear it may be because of adding the value on the map. My idea for tackling that is to try changing the TrieLabel into an enum instead of a struct. Then a TrieLabel::Value will mean its parent is the last Label of that entry.