search

thinkeridea / go-extend

go语言扩展包，收集一些常用的操作函数，辅助更快的完成开发工作，并减少重复代码
https://pkg.go.dev/github.com/thinkeridea/go-extend
MIT License
1.34k stars 160 forks source link

动态评估bytes.Buffer的需求的容量，全局共享bytes.Buffer池 #16

Closed thinkeridea closed 3 years ago

thinkeridea commented 3 years ago

采用全局共享不同容量区间（采用2的幂次方来划分），一共 22 个区间，最小64 byte， 最大 256 MB， 根据动态计算，在获取 bytes.Buffer 时从相应桶的 sync.Pool 来获取，以减少多余的内存浪费及bytes.Buffer 扩容次数。

相较于使用固定容量的 sync.Pool 在需求容量动态波动时可以获取30~40倍的性能提升（这并不包含GC带来的性能提升，那将是更令人兴奋的），在内存分配上可以减少近300倍。

即使在每次都获取固定容量，且不发生扩容的情况（这会导致动态池只是用单个桶）， sync.Pool并没有带来很大的惊喜，动态池相较于 sync.Pool 性能略下降 28%， 单次约 17ns， 即使这样这依然让人振奋，大多数我们都在处理不同大小的数据，使用动态bytes.Buffer共享池我们将收益巨大。

goos: darwin
goarch: amd64
pkg: github.com/thinkeridea/go-extend/pool/benchmark
BenchmarkBufferPool
BenchmarkBufferPool-8                    1088794              1102 ns/op             387 B/op          0 allocs/op
BenchmarkBufferSyncPool
BenchmarkBufferSyncPool-8                  49735             37895 ns/op          115277 B/op          0 allocs/op
BenchmarkBufferFixedSizePool
BenchmarkBufferFixedSizePool-8          27296764                42.7 ns/op             0 B/op          0 allocs/op
BenchmarkBufferFixedSizeSyncPool
BenchmarkBufferFixedSizeSyncPool-8      26099842                59.6 ns/op           192 B/op          0 allocs/op
PASS
ok      github.com/thinkeridea/go-extend/pool/benchmark 8.627s
codecov[bot] commented 3 years ago

Codecov Report

Merging #16 (774c519) into main (b27a4fe) will increase coverage by 0.00%. The diff coverage is 98.50%.

Impacted file tree graph

@@           Coverage Diff           @@
##             main      #16   +/-   ##
=======================================
  Coverage   90.77%   90.77%           
=======================================
  Files          25       26    +1     
  Lines         791      802   +11     
=======================================
+ Hits          718      728   +10     
- Misses         51       52    +1     
  Partials       22       22
Impacted Files Coverage Δ
pool/std_buffer.go 75.00% <75.00%> (ø)
exstrings/join_int.go 100.00% <100.00%> (ø)
pool/buffer.go 100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update b27a4fe...774c519. Read the comment docs.