The prefix search as currently done in GreatestLowerBoundIterator is a clumsy data structure despite its recent improvement. We need a true trie data structure.
Requirements:
Fast checking of membership.
Can be updated with new prefixes.
More precision: The complexity of a trie search is linear in the length of the key being tested.
The complexity of a single search to GreatestLowerBoundIterator is O(log(M) N) with N the length of the key being tested and M the number of prefixes in the database. This complexity is the one we are exposed to in get and multi_get.
However for the function call for_each_key_while and similar, the complexity is better since the keys being called are lexicographically ordered and so the search is more efficient in that case.
Hello @MathieuDutSik I think the description of the issue needs to be slighlty changed after #1381 i.e. changing fromGreatestLowerBoundIterator to SuffixClosedSetIterator
The prefix search as currently done in
GreatestLowerBoundIterator
is a clumsy data structure despite its recent improvement. We need a truetrie
data structure.Requirements:
More precision: The complexity of a
trie
search is linear in the length of the key being tested.The complexity of a single search to
GreatestLowerBoundIterator
isO(log(M) N)
withN
the length of the key being tested andM
the number of prefixes in the database. This complexity is the one we are exposed to inget
andmulti_get
. However for the function callfor_each_key_while
and similar, the complexity is better since the keys being called are lexicographically ordered and so the search is more efficient in that case.