Closed Dieterbe closed 8 years ago
for the record, i just tried with the below patch applied which uses 1M points per op, to make sure the issue isn't related to the small amount of data being used in the benchmarks, but the output is similar (actually decodes are slower)
go test -run=X -bench=. -benchmem ⏎
PASS
BenchmarkEncode-8 10 162941655 ns/op 73.65 MB/s 45185747 B/op 71 allocs/op
BenchmarkDecode-8 20 93936141 ns/op 127.75 MB/s 16883924 B/op 14 allocs/op
diff --git a/tsz_test.go b/tsz_test.go
index 7bca3a6..fb33964 100644
--- a/tsz_test.go
+++ b/tsz_test.go
@@ -2,6 +2,7 @@ package tsz
import (
"github.com/dgryski/go-tsz/testdata"
+ "math/rand"
"testing"
"time"
)
@@ -185,22 +186,27 @@ func testConcurrentRoundtrip(t *testing.T, sleep time.Duration) {
}
func BenchmarkEncode(b *testing.B) {
- b.SetBytes(int64(len(testdata.TwoHoursData) * 12))
+ in := make([]testdata.Point, 1000000)
+ for i := 0; i < len(in); i++ {
+ in[i] = testdata.Point{rand.Float64(), uint32(i)}
+ }
+ b.SetBytes(int64(len(in) * 12))
+ b.ResetTimer()
+
for i := 0; i < b.N; i++ {
- s := New(testdata.TwoHoursData[0].T)
- for _, tt := range testdata.TwoHoursData {
+ s := New(in[0].T)
+ for _, tt := range in {
s.Push(tt.T, tt.V)
}
}
}
func BenchmarkDecode(b *testing.B) {
- b.SetBytes(int64(len(testdata.TwoHoursData) * 12))
- s := New(testdata.TwoHoursData[0].T)
- for _, tt := range testdata.TwoHoursData {
- s.Push(tt.T, tt.V)
+ s := New(0)
+ for i := 0; i < 1000000; i++ {
+ s.Push(uint32(i), rand.Float64())
}
-
+ b.SetBytes(int64(12 * 1000 * 1000))
b.ResetTimer()
for i := 0; i < b.N; i++ {
LGTM
I didn't realize until recently that decoding (and encoding) is so expensive. for comparison, in metric-tank - after some optimisations - data consolidation works at 2~6GB/s and json encoding is around 50 MB/s (luckily this is post-consolidation)
@dgryski can you have a look and see if there's any way to make decoding faster? here's a screenshot of a profile I took the other day: http://i.imgur.com/7tTIFHd.png readBits and readByte seem to be a bottleneck for us. thanks