Closed MighTguY closed 4 years ago
Of course we could use a Trie instead of a Dictionary/Map for holding the deletes. But I don't think there will be a benefit, neither in lookup speed nor in memory consumption. But I will do a quick experiment to verify that.
Test results: 500.000 terms from frequency_dictionary_en_500_000.txt resulting in 12.577.734 deletes:
Radix trie<string,long>: 1800 MB
Dictionary<string,long>: 1366 MB
Dictionary<uint hash, long>: 600 MB //instead of key=term I use key=FNV-1a-Hash(term)
The dictionary is more space efficient than the trie. For auto-complete I use a trie because it allows prefix range search. But for SymSpell spelling correction I just need a simple key lookup.
There are ways to optimize the space efficiency of tries, e.g. Succinct Tries . But those are slow.
Hi @wolfgarbe , Greetings Currently, we use Dictionary/Map for holding data? can we move it to a more optimized trie?
similar to your repo PruningRadixTrie (https://github.com/wolfgarbe/PruningRadixTrie)