Closed okayzed closed 4 years ago
The first attempt is a bit hacky, but this significantly speeds up queries that ask for a subset of columns!
5b9eab2
When I try this on jaeger spans from redbull, it dropped the query time of 138mm records from 36s to 12s when asking for a single column. Asking for two int columns takes 18 - 20s, but if a group by is added, it can significantly slow the query down.
I'm super excited about this performance and I find it quite surprising - I think it has to do with lowering the 1) record reset time and 2) improving cache locality.
https://github.com/logv/sybil/tree/shorten_key_table has current work. in testing but will be merged into master after its ready
Right now, a Record has .Ints and .Strs allocated as arrays of size max_key_id. This is suboptimal and slow
Need to figure out a way to only allocate the number of requested columns for the record.