Closed mithro closed 5 years ago
I found this to be a bottleneck and translated https://github.com/julian-klode/triehash to Python and integrated it with the program. It really speeds things up!
10 runs of the previous commit with some rr_graph(a month ago):
13.62 13.44 13.46 13.53 13.44 13.50 13.45 13.54 13.63
10 runs of the commit with tries:
9.43 9.67 9.38 9.46 9.49 9.34 9.39 9.49 9.36 9.43
Now, this implementation of trie depends on the fact that unaligned 64-bit accesses are not very slow in amd64 processors. However, it should be possible to add a knob and provide "flat" tries(checks every character separately) or strcmp chains for processors with slow unaligned access.
This is done, with the remaining issue #21.
Currently you use the follow structure a lot;
Do some profiling and figure out if there is a better way to do this.
strcmp
might actually turn out to be pretty fast here...As doing this shouldn't change the external interface, so it is something we can leave till later.