opensearch-project / opensearch-go

Go Client for OpenSearch
https://opensearch.org/docs/latest/clients/go/
Apache License 2.0
188 stars 96 forks source link

[FEATURE] Use pooled gzip writer for request body gzip compression #520

Closed pakio closed 2 months ago

pakio commented 2 months ago

Is your feature request related to a problem?

While I was profiling the performance of our application, I noticed that gzip.NewWriter and extending buffer space are consuming a of memory. This is a feature request to update logic to pool Writer object and buffer for better performance, similar to other package, e.g, grpc-go, go-ethereum.

What solution would you like?

Use sync.Pool to share gzip writer object

What alternatives have you considered?

N/A

Do you have any additional context?

Here's the benchmark I run while I was contributing to other package, comparing performance of gzip-ing {"query":{"match_all":{}}}. Result might differ depending on the use case but I expect to see similar impact by the update.

=== RUN   BenchmarkTransport/Compress_body_(pool:_false)
BenchmarkTransport/Compress_body_(pool:_false)
BenchmarkTransport/Compress_body_(pool:_false)-8                    9450            117462 ns/op          815014 B/op         31 allocs/op
=== RUN   BenchmarkTransport/Compress_body_(pool:_true)
BenchmarkTransport/Compress_body_(pool:_true)
BenchmarkTransport/Compress_body_(pool:_true)-8                    57690             20462 ns/op            1152 B/op         12 allocs/op