n-young / trustdb

0 stars 1 forks source link

Optimizations on how we store Records in Series => Compression #18

Closed n-young closed 3 years ago

n-young commented 3 years ago

Right now in Series, we store Records in their completely raw form - this is bad since it creates a lot of duplication (the name/labels are the same in every Record in a Series, the Metric Keys are all the same as well). We should store Records in a new form, like a SeriesRecord, that only has something like the following form:

struct SeriesRecord {
    metrics: Vec<f64>
    timestamp: u64
}

Since we can infer the metrics by doing them in lexicographical sorted order, a timestamp should be stored in an int to allow compression there, and the name/labels will all be the same.