reduce memory allocations when zlib compression is enabled

[dvilaverde]

When enabling compression in our production application we're seeing a fairly large increase in memory allocations and GC duration. Based on some profiling I was able to narrow it down to the new creation of the zlib writers and readers that are thrown away after every packet.

I've implemented re-use of the writers using sync.Pool along with a few other changes, such as not creating a zilb Writer when the packet data size is less then minCompressionLength to help alleviate the pressure we see on the garbage collector.

Here is a benchmark for before my changes:

go test ./pkg/database/maria/testserver -cpu 1,2,4 -bench=. -run=^# -benchtime=1s -benchmem
goos: darwin
goarch: arm64
pkg: git.company.com/server/test/pkg/database/maria/testserver
BenchmarkInsert_TestServer           664           1762367 ns/op         1529787 B/op        161 allocs/op
BenchmarkInsert_TestServer-2        2130            515707 ns/op         1529917 B/op        162 allocs/op
BenchmarkInsert_TestServer-4        2086            501147 ns/op         1530396 B/op        163 allocs/op
PASS
ok      git.company.com/server/test/pkg/database/maria/testserver     5.038s

and this is the benchmark after the changes.

go test ./pkg/database/maria/testserver -cpu 1,2,4 -bench=. -run=^# -benchtime=1s -benchmem
goos: darwin
goarch: arm64
pkg: git.company.com/server/test/pkg/database/maria/testserver
BenchmarkInsert_TestServer          2968            346505 ns/op            8734 B/op        101 allocs/op
BenchmarkInsert_TestServer-2        3057            364719 ns/op            9099 B/op        101 allocs/op
BenchmarkInsert_TestServer-4        3032            375451 ns/op           13800 B/op        101 allocs/op
PASS
ok      git.company.com/server/test/pkg/database/maria/testserver     4.951s

I've been running my for in production for a few days now making SQL calls at a rate of about 1000/s and the application has been much more stable and we've been able to scale down our cluster size due to the fewer CPU cycles we're spending doing GC.

[lance6716]

thanks! I'll take a look next week.

[dvilaverde]

rest lgtm. Thanks!

I've address your feedback. Thanks for taking the time to look at this.