high compression seems only achievable in special cases like zeroes,
equal values, floats that are integers. In "real" float scenarios,
compression is marginal.
chunk size only matters in those cases, and there, around 120 points
seems like the sweet spot (like the paper said). However, "noisy"
data still compresses poorly and chunksize does not seem to matter.
raw data below:
BenchmarkEncodeSize10min-8 0 0 ns/op
--- BENCH: BenchmarkEncodeSize10min-8
tsz_test.go:275: encode 10 all zeroes float points: 25 Bytes. 2.50 B/point
tsz_test.go:275: encode 10 all the same float points: 25 Bytes. 2.50 B/point
tsz_test.go:275: encode 10 small range pos int points: 40 Bytes. 4.00 B/point
tsz_test.go:275: encode 10 small range pos float points: 78 Bytes. 7.80 B/point
tsz_test.go:275: encode 10 large range pos float points: 81 Bytes. 8.10 B/point
tsz_test.go:275: encode 10 random positive float points: 84 Bytes. 8.40 B/point
tsz_test.go:275: encode 10 random pos/neg float points: 90 Bytes. 9.00 B/point
BenchmarkEncodeSize30min-8 0 0 ns/op
--- BENCH: BenchmarkEncodeSize30min-8
tsz_test.go:275: encode 30 all zeroes float points: 30 Bytes. 1.00 B/point
tsz_test.go:275: encode 30 all the same float points: 30 Bytes. 1.00 B/point
tsz_test.go:275: encode 30 small range pos int points: 71 Bytes. 2.37 B/point
tsz_test.go:275: encode 30 small range pos float points: 213 Bytes. 7.10 B/point
tsz_test.go:275: encode 30 large range pos float points: 218 Bytes. 7.27 B/point
tsz_test.go:275: encode 30 random positive float points: 237 Bytes. 7.90 B/point
tsz_test.go:275: encode 30 random pos/neg float points: 259 Bytes. 8.63 B/point
BenchmarkEncodeSize1h-8 0 0 ns/op
--- BENCH: BenchmarkEncodeSize1h-8
tsz_test.go:275: encode 60 all zeroes float points: 38 Bytes. 0.63 B/point
tsz_test.go:275: encode 60 all the same float points: 38 Bytes. 0.63 B/point
tsz_test.go:275: encode 60 small range pos int points: 116 Bytes. 1.93 B/point
tsz_test.go:275: encode 60 small range pos float points: 416 Bytes. 6.93 B/point
tsz_test.go:275: encode 60 large range pos float points: 424 Bytes. 7.07 B/point
tsz_test.go:275: encode 60 random positive float points: 485 Bytes. 8.08 B/point
tsz_test.go:275: encode 60 random pos/neg float points: 510 Bytes. 8.50 B/point
BenchmarkEncodeSize2h-8 0 0 ns/op
--- BENCH: BenchmarkEncodeSize2h-8
tsz_test.go:275: encode 120 all zeroes float points: 53 Bytes. 0.44 B/point
tsz_test.go:275: encode 120 all the same float points: 53 Bytes. 0.44 B/point
tsz_test.go:275: encode 120 small range pos int points: 204 Bytes. 1.70 B/point
tsz_test.go:275: encode 120 small range pos float points: 813 Bytes. 6.78 B/point
tsz_test.go:275: encode 120 large range pos float points: 824 Bytes. 6.87 B/point
tsz_test.go:275: encode 120 random positive float points: 980 Bytes. 8.17 B/point
tsz_test.go:275: encode 120 random pos/neg float points: 1013 Bytes. 8.44 B/point
BenchmarkEncodeSize6h-8 0 0 ns/op
--- BENCH: BenchmarkEncodeSize6h-8
tsz_test.go:275: encode 360 all zeroes float points: 113 Bytes. 0.31 B/point
tsz_test.go:275: encode 360 all the same float points: 113 Bytes. 0.31 B/point
tsz_test.go:275: encode 360 small range pos int points: 559 Bytes. 1.55 B/point
tsz_test.go:275: encode 360 small range pos float points: 2466 Bytes. 6.85 B/point
tsz_test.go:275: encode 360 large range pos float points: 2447 Bytes. 6.80 B/point
tsz_test.go:275: encode 360 random positive float points: 2960 Bytes. 8.22 B/point
tsz_test.go:275: encode 360 random pos/neg float points: 3023 Bytes. 8.40 B/point
BenchmarkEncodeSize12h-8 0 0 ns/op
--- BENCH: BenchmarkEncodeSize12h-8
tsz_test.go:275: encode 720 all zeroes float points: 203 Bytes. 0.28 B/point
tsz_test.go:275: encode 720 all the same float points: 203 Bytes. 0.28 B/point
tsz_test.go:275: encode 720 small range pos int points: 1091 Bytes. 1.52 B/point
tsz_test.go:275: encode 720 small range pos float points: 4945 Bytes. 6.87 B/point
tsz_test.go:275: encode 720 large range pos float points: 4882 Bytes. 6.78 B/point
tsz_test.go:275: encode 720 random positive float points: 5930 Bytes. 8.24 B/point
tsz_test.go:275: encode 720 random pos/neg float points: 6038 Bytes. 8.39 B/point
BenchmarkEncodeSize24h-8 0 0 ns/op
--- BENCH: BenchmarkEncodeSize24h-8
tsz_test.go:275: encode 1440 all zeroes float points: 383 Bytes. 0.27 B/point
tsz_test.go:275: encode 1440 all the same float points: 383 Bytes. 0.27 B/point
tsz_test.go:275: encode 1440 small range pos int points: 2156 Bytes. 1.50 B/point
tsz_test.go:275: encode 1440 small range pos float points: 9904 Bytes. 6.88 B/point
tsz_test.go:275: encode 1440 large range pos float points: 9753 Bytes. 6.77 B/point
tsz_test.go:275: encode 1440 random positive float points: 11870 Bytes. 8.24 B/point
tsz_test.go:275: encode 1440 random pos/neg float points: 12068 Bytes. 8.38 B/point
@dgryski @woodsaj my personal conclusions: