Note: this issue is NOT about HTTP range requests (see #98).
Right now our query system only supports strict equivalence for meta-data filters. We should normalize to be case insensitive. More importantly, we should allow <, <=, >, >= operations.
The problem is that when using an ordered KV-store, doing a prefix lookup gives us the results in order, which we rely on (the table we use here is SLNFieldValueAndMetaFileID). If we do a range lookup, the results (meta-file ID) are in effectively random order.
The solution is a higher order index, but I'm not sure which one or how to implement it. Maybe r-trees. It still has to be ACID and write-optimized, so we should find a way to build it on top of our existing database layer.
Note: this issue is NOT about HTTP range requests (see #98).
Right now our query system only supports strict equivalence for meta-data filters. We should normalize to be case insensitive. More importantly, we should allow
<,<=,>,>=operations.The problem is that when using an ordered KV-store, doing a prefix lookup gives us the results in order, which we rely on (the table we use here is
SLNFieldValueAndMetaFileID). If we do a range lookup, the results (meta-file ID) are in effectively random order.The solution is a higher order index, but I'm not sure which one or how to implement it. Maybe r-trees. It still has to be ACID and write-optimized, so we should find a way to build it on top of our existing database layer.