search

That3Percent / tree-buf

An experimental serialization system written in Rust
MIT License
256 stars 8 forks source link

Question about array handling #32

Open jgarvin opened 1 year ago

jgarvin commented 1 year ago

I think the separate compression for each path is interesting but I couldn't tell from the README how it gets applied to arrays. If I have a sequence of messages and there is an array field, is every index of the array considered the leaf of a different path, or are the arrays concatenated for purposes of compression or ...?

That3Percent commented 1 year ago

The arrays are concatenated for purposes of compression. There is a separate path for the length of the arrays to compress as well. So, imagine you have [[0, 1, 2], [2, 2, 3], [1]]. This becomes: lengths: [3, 3, 1], data: [0, 1, 2, 2, 2, 3, 1].