search

thanos-io / promql-engine

Multi-threaded PromQL engine implementation based on the Volcano paper.
Apache License 2.0
133 stars 52 forks source link

plan,execution: late coalesce experiments #372

Open MichaHoffmann opened 6 months ago

MichaHoffmann commented 6 months ago

So far only a sketch to get a feeling; benchmark ( if i get rid of batch size! ) looks fairly promising though:

$ go test . -run xxx -bench BenchmarkRangeQuery/experiment/new_engine -v -test.count 5
goos: linux
goarch: amd64
pkg: github.com/thanos-io/promql-engine/engine
cpu: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz
BenchmarkRangeQuery
BenchmarkRangeQuery/experiment
BenchmarkRangeQuery/experiment/new_engine
BenchmarkRangeQuery/experiment/new_engine-8                   31      35560327 ns/op     8950556 B/op      70752 allocs/op
BenchmarkRangeQuery/experiment/new_engine-8                   34      34607836 ns/op     8947958 B/op      70744 allocs/op
BenchmarkRangeQuery/experiment/new_engine-8                   36      33039457 ns/op     8930715 B/op      70658 allocs/op
BenchmarkRangeQuery/experiment/new_engine-8                   37      31686100 ns/op     8918604 B/op      70638 allocs/op
BenchmarkRangeQuery/experiment/new_engine-8                   36      32301408 ns/op     8930411 B/op      70669 allocs/op
PASS
ok      github.com/thanos-io/promql-engine/engine   17.815s

vs on main 

$ go test . -run xxx -bench BenchmarkRangeQuery/experiment/new_engine -v -test.count 5
goos: linux
goarch: amd64
pkg: github.com/thanos-io/promql-engine/engine
cpu: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz
BenchmarkRangeQuery
BenchmarkRangeQuery/experiment
BenchmarkRangeQuery/experiment/new_engine
BenchmarkRangeQuery/experiment/new_engine-8                   22      46970488 ns/op     6916236 B/op      39532 allocs/op
BenchmarkRangeQuery/experiment/new_engine-8                   25      46018216 ns/op     6908259 B/op      39510 allocs/op
BenchmarkRangeQuery/experiment/new_engine-8                   24      49707030 ns/op     7131612 B/op      39635 allocs/op
BenchmarkRangeQuery/experiment/new_engine-8                   24      50225178 ns/op     6981826 B/op      39560 allocs/op
BenchmarkRangeQuery/experiment/new_engine-8                   21      52848319 ns/op     7079423 B/op      39609 allocs/op
PASS
ok      github.com/thanos-io/promql-engine/engine   16.435s

To get to a point where we can do it we should do a few preparatory PRs: