HdrHistogram / hdrhistogram-go

A pure Go implementation of Gil Tene's HDR Histogram.
MIT License
429 stars 64 forks source link

Optimized windowed histogram for large window size. #26

Closed pamburus closed 2 years ago

pamburus commented 6 years ago

Added new benchmark for typical scenario of usage of WindowedHistogram (rotate and merge every tick) with a large window of 100 histograms with 100k values in each: BenchmarkWindowedHistogramRotateAndMerge

Result on my MacBook Pro 13 before optimization:

BenchmarkHistogramRecordValue-4                 100000000           16.2 ns/op         0 B/op          0 allocs/op
BenchmarkNew-4                                    200000          8779 ns/op       65632 B/op          2 allocs/op
BenchmarkWindowedHistogramRecordAndRotate-4     100000000           17.2 ns/op         0 B/op          0 allocs/op
BenchmarkWindowedHistogramMerge-4                 100000         13211 ns/op           0 B/op          0 allocs/op
BenchmarkWindowedHistogramRotateAndMerge-4           200       8737304 ns/op           0 B/op          0 allocs/op
PASS
ok      github.com/pamburus/hdrhistogram    12.370s

Result on my MacBook Pro 13 after optimization:

BenchmarkHistogramRecordValue-4                 100000000           15.9 ns/op         0 B/op          0 allocs/op
BenchmarkNew-4                                    200000          8100 ns/op       65632 B/op          2 allocs/op
BenchmarkWindowedHistogramRecordAndRotate-4     100000000           17.0 ns/op         0 B/op          0 allocs/op
BenchmarkWindowedHistogramMerge-4                 200000         10052 ns/op           0 B/op          0 allocs/op
BenchmarkWindowedHistogramRotateAndMerge-4         10000        183992 ns/op           0 B/op          0 allocs/op
PASS
ok      github.com/pamburus/hdrhistogram    44.864s

Before optimization: 8737304 ns/op (8.7 ms) After optimization: 183992 ns/op (0.2 ms)

Note that calling Merge after Rotate is not needed anymore. Rotate now returns already merged histogram, using it's result provides ability to achieve maximum performance. Merge still can be used at any time to get immediate representation of the whole histogram, including current histogram which has not been rotated yet. But there is no much practical use of this function.

pamburus commented 2 years ago

Closed since already merged