search

bookingcom / nanotube

High-performance router for Graphite.
Apache License 2.0
56 stars 12 forks source link

Reduce memory allocation due to string copying while processing records. #139

Closed grzkv closed 2 years ago

grzkv commented 2 years ago

Currently, we allocate a lot of memory during processing, see pic v

image

This likely has negative impact on performance.

We can avoid that. For example, we could re-use already allocated strings.

grzkv commented 2 years ago

This was significantly improved by #141

Here's the memory allocation profile for the same benchmark.

Done for strings (before the #141) https://github.com/bookingcom/nanotube/blob/278c1dffc08d087acff886f72e9b73aaec4eb3cc/cmd/nanotube/process_test.go#L320

image

Done for []byte (after the #141 ) https://github.com/bookingcom/nanotube/blob/278c1dffc08d087acff886f72e9b73aaec4eb3cc/cmd/nanotube/process_test.go#L287

image

The allocation went down from 1.3 GB to 0.3 GB.